Development of the Student Experience of Teacher Support Scale: Measuring the Experience of Children Who Report Aggression and Bullying

  • Helen J. NelsonEmail author
  • Garth E. Kendall
  • Sharyn K. Burns
  • Kimberly A. Schonert-Reichl
  • Robert T. Kane
Original Article


Social isolation of school children is increased when teachers feel unable to respond to children’s report of witnessing or experiencing aggression at school. Teachers can find it difficult to respond adequately to children who report aggression when the aggression is deliberately hidden from the teacher by the perpetrator. This study aimed to explore, and then confirm, the latent factor structure of the Student Experience of Teacher Support Scale (SETSS), a new instrument designed to measure children’s experience when reporting aggression, including aggression that may have been cleverly hidden from school teachers. This paper reports two phases of the study (grades 4 to 6, Perth, Western Australia): sample 1 (N = 174) and sample 2 (N = 642). Qualitative methods informed instrument design. This paper describes the quantitative phases: first, exploratory factor analysis identified a two-factor model, student experience and heard by the teacher; second, confirmatory factor analysis and construct validity were conducted. The SETSS has a two-factor model of sound psychometric fit. The teacher’s response to children who report aggression can be experienced by children as supportive; alternatively, children may experience increased harm. The harm experienced by victims of aggression is increased when aggression is deliberately and intentionally hidden from teachers by the perpetrator. This adds to the empirical understanding of factors that predict bullying, the identification of harm that results from bullying, and will help track the effectiveness of interventions and prevention and management of this significant health threat.


Elementary school aggression teacher support 


Aggressive behavior has two features, “intent to harm” by the perpetrator and “a feeling of hurt” by the victim (Bovaird 2010, p. 278). Bullying, a subset of aggression, is the proactive and repeated use of aggressive behavior by a powerful child over a less powerful child to achieve a goal (Volk et al. 2017). Bullying is challenging to measure. The difficulty of measurement has been attributed to the inadequate measurement of power imbalance and the fact that children often intentionally and deliberately hide aggressive behavior from authority figures (Volk et al. 2017). This paper reports on the development of an instrument to measure children’s own experience of encountering and witnessing aggression, and reporting it to the teacher (Nelson et al. 2015). The scale measures children’s experience of teacher support after the teacher has been told. This scale is particularly relevant when aggression is covert; that is, it is deliberately and intentionally hidden from the teacher by the perpetrator or by peers in the form of bystander behavior (Cross et al. 2009). We contend that this form of aggression contributes to the power imbalance that is experienced by children who are bullied. Much school-based aggression and bullying are hidden from the teacher (Rodkin et al. 2015; Volk et al. 2017). For this reason, covert aggression is discussed in this paper in conjunction with bullying.

The ages of 9 to 12 years are a key period for the development of relational aggression as children’s goals of popularity and social dominance peak (De Laet et al. 2015). Vaillancourt and Hymel (2006) comment that the term dominance is traditionally applied to people who manipulate the behavior of others; this relates to social power. Children who achieve a goal of social dominance by manipulating the peer group are often perceived as popular. They unite peers through the strategic use of intimacy and social exclusion (Kiefer and Wang 2016). Many children who have social power are skilled at intentionally hiding aggressive behavior from others and may even be “seen in a positive light by teachers” (Hawley 2003, p. 301). For this reason, social power is often not visible to adults, increasing the power over the bullied child (Rosen et al. 2017). For example, perpetrators of bullying have reported that their reputation was enhanced when adults in authority ignored children’s reports of being bullied (Houghton et al. 2012). Similarly, children have reported increased harm when bullying was hidden from teachers by children who skillfully lied to the teacher to escape detection (Nelson et al. 2018).

Recent studies have shown improved peer support for children who are bullied when teachers provide emotional support to those who report the bullying (Norwalk et al. 2016; Serdiouk et al. 2016). However, teachers have difficulty responding to children’s reports of aggression and bullying when they have not witnessed the behavior (Mishna et al. 2006). Teachers have reported that bullying is becoming an increasingly complex behavior that often remains hidden (Cunningham et al. 2016). When teachers do try and uncover the behavior, children’s skills of deception can result in the aggression going “deeper down” and teacher’s efforts to resolve the situation “can actually make it worse” (Cunningham et al. 2016, p. 466). This implies that the power imbalance between the child who is bullying and the bullied child is increased as the bullied child is left exposed to continuing victimization.

The authors of one early study reported that teachers intervene in bullying situations in only 15–18% of cases (Craig et al. 2000). This may be because teachers are not aware of the bullying, but it may be that they are aware and fail to take action (Veenstra et al. 2014). In more recent studies, it has been found that students are concerned that if they report bullying, they may not be believed, or they could make the situation worse (Bradshaw et al. 2007; Cross et al. 2009). Teachers have similarly reported that their efforts to reduce bullying can actually make it worse (Cunningham et al. 2016). The failure of teachers to intervene has been attributed to the complexity of bullying (Cunningham et al. 2016), teacher fatigue (Swift et al. 2017), and a lack of awareness as to how best to intervene (Cross et al. 2011).

Importantly, most teachers understand the importance of children’s social development and child-teacher relationships (Day et al. 2006). The social environment of the classroom is integral to children’s learning outcomes, and to the happiness and professional identity of teachers (Day et al. 2006; Swan and Riley 2015). However, the increasing complexity of aggression and bullying places a burden on teachers as they seek to meet the social needs of children along with curriculum demands (Cunningham et al. 2016). Despite the attention given to the issue by academics and the popular press, it has been suggested that teachers’ influence on peer relationships and peer aggression is understudied (Ertesvåg 2016; Gest and Rodkin 2011; Mucherah et al. 2018).

It was recently found that bullying behavior and victimization scores were lower among secondary school students (n = 2273) in Kenya when teachers actively intervened (Mucherah et al. 2018). Mucherah et al. (2018) simply asked students if the teacher had intervened; they did not ask how the teacher had intervened. We believe that a better understanding of students’ experience of the support teachers provide when they are told about an incident is essential to inform interventions to help teachers effectively manage bullying. In this paper, we report the empirical evaluation of the Student Experience of Teacher Support Scale (SETSS), a new instrument designed to measure the experience of children who have told the teacher about an incident of aggressive behavior.

The study was influenced and informed by developmental systems theory (Bronfenbrenner and Morris 2006). According to this theory, the classroom is a microsystem in which children’s development is influenced by their mutual interaction with the people in their immediate environment (Bronfenbrenner 2001). The design of the SETSS was informed by thematic analysis of focus group discussions with children aged 9 to 11 years (Nelson et al. 2018). The inclusion of children in the development of the instrument is in accord with the United Nations Convention on the Rights of the Child (UNICEF 1989). Specifically, children have the right to express their own views in all matters that affect them and to participate in processes that aim to bring about change (Davey et al. 2010).

The first objective (phase 1) of the current research was to explore the latent factor structure of the SETSS. The instrument was designed to include items that might reflect children’s experience of reporting aggression that was intentionally hidden from school teachers. The second objective (phase 2) was to confirm the factor structure and examine the reliability and construct validity of the SETSS.

Materials and Methods


Participants were students in grades 4 to 6 (aged 8 to 12) from a purposive sample of non-government schools in Perth, Western Australia, where the tuition fees are set at a moderate level (n = 462, 46% male, 54% female). The schools were represented on average by families of socio-educational advantage; children from less advantaged families did however attend the schools (Index of Community Socio-Educational Advantage in Australia; ACARA 2015). Families of cultural and ethnic diversity were represented; 13% of participants in phase 1 and 24% of those in phase 2 spoke a language other than English at home.

Phase 1

All students who had not completed the questionnaire during instrument development from grades 4 to 6 from one school (N = 174) were invited to participate via a letter sent home by the school. Active parental/guardian consent and student assent were provided for 121 participants. Of these, 111 students (64%) were present on the day of data collection and completed an online questionnaire (41% male, 59% female). The minimum number of participants recommended for factor analysis is five per variable (Russell 2002). The instrument under investigation consisted of 13 items and was answered by 69 participants who answered “yes” to a stem question and subsequently answered the new instrument. The present sample size was therefore adequate for exploratory factor analysis. Information about instrument development is presented in the “Measures” section.

Phase 2

Schools were purposively selected for phase 2 by approaching principals (N = 10) from moderate fee-paying non-government schools in the metropolitan region of Perth, Western Australia. Of these, four principals agreed for their school to participate, four declined (three because the schools were currently participating in different research projects, one due to new appointment to the school as principal), and two principals did not respond. All students who had not previously completed the questionnaire were invited to participate (N = 642) via a letter sent home by each school. Informed parental/guardian consent was received for 351 students (55%), 14 students were not at school on the day of data collection, and one student did not give assent. Therefore, in total, 337 students (52% of the sample) completed the online survey (51% male, 49% female). This sample size was adequate for confirmatory factor analysis and multigroup analysis (Byrne 2012). Parental/guardian consent was given for students from one participating school to complete a retest after a 2-week interval (n = 58); the instrument was completed a second time by 50 students (86%). Students completing the test-retest were allocated an identifying number; all other data collection was anonymous.


Ethics approval was obtained from the University Human Research Ethics Committee; and the study was performed in accordance with the ethical standards as laid down in the National Statement on Ethical Conduct in Human Research (the National Health and Medical Research Council, the Australian Research Council and the Australian Vice-Chancellors’ Committee 2007). The first author met with a nominated person from each school to explain the process and to make provisions to lessen the risk of harm to participants. Data collection took place in November 2015 (phase 1) and June 2016 (phase 2). The first author was present during data collection and used simple language to explain the research to children before they simultaneously answered the anonymous online questionnaire on a computer or tablet device. The questionnaire included self-report measures of aggression and power imbalance and was introduced as a survey “to find out how kids treat each other at school.” Participants were informed that the researchers would not discuss children’s answers with teachers or parents. A research assistant was present to answer questions or to help students with reading. A research diary was kept, and a record was made of any questions asked by participants.


Phase 1

The Student Experience of Teacher Support Scale

In phase 1, the new instrument comprised 13 items to measure children’s experiences when they told a teacher of aggressive behavior that was directed toward them or someone else. Items were identified through focus group discussion with children, thematic coding included a search of the prevalent attitudes and views expressed by participants (Nelson et al. 2018). Each item in the new instrument reflected the language used by focus group participants, for example “The other student got away with it by telling a lie” (items are listed in Table 1). The children who had participated in focus groups assessed face validity of the instrument. Expert reviewers (with expertise in psychology, education, and public health) universally agreed on the content validity of the instrument at scale level (Polit and Beck 2006). The instrument was displayed in the online questionnaire using the display logic function when children answered “yes” to the question “Have you ever told a teacher when other students have been mean to you or someone else on purpose.” The response (Click if this has happened to you) was coded as a binary outcome, 0 (no) or 1 (yes) as recommended by expert reviewers.
Table 1

EFA frequencies and factor loadings of the SETSS


Frequency (n = 69)







Item 1





The teacher thought it was just an accident

Item 2





It got solved

Item 3





The other student got away with it by telling a lie

Item 4





The teacher wasn’t able to help

Item 5





The teacher helped me

Item 6





The teacher thought we should ignore it

Item 7





Telling the teacher made it worse

Item 8





Telling the teacher made it better

Item 9





The teacher did not believe me

Item 10





Everyone just got into way more trouble

Item 11





My friends excluded me because I told the teacher

Item 12





The teacher thought it did not happen

Item 13





The student who was mean to me is popular with the teachers, so nothing was done about it

Note: EFA factor loading in SPSS, values below 0.30 are suppressed. Children answered as many items as were relevant to their experience

Phase 2

The Student Experience of Teacher Support Scale

The SETSS was slightly modified for phase 2; the response scale remained unchanged. In response to the research diary made in phase 1 of data collection, and in consultation with an expert reviewer, the language of some items was changed for phase 2 (items are listed in Table 2). Items 4 and 7 were removed from the experience factor, item 4 measured the equivalent concept of item 5, and item 7 measured the equivalent concept of item 8. An eight-item factor resulted. In phase 1, the eight-item experience factor had a strong fit (α = .856, CR = .925) and the three-item heard factor showed adequate fit (α = .736, CR = .885). A minimum of four items on each scale is recommended to identify an effective factor (Raykov 1997; Russell 2002). An item, “The teacher listened to me,” was added to the three-item heard factor. The item content validity index (I-CVI) was then reviewed by expert reviewers (seven teachers, two school psychologists), and each item was within the recommended range of .78 to 1 (Polit and Beck 2006). The SETSS was included in a broader questionnaire to which children had reported on their experience “in the last few months.”
Table 2

CFA factor loadings for the SETSS


Frequency (n = 230)

Factor loadings





1.The teacher said it was just an accident





2. It got solved





3. The other student got away with it by telling a lie





4. The teacher helped me





5. The teacher said we should forget about it





6. Telling the teacher made it better





7. The teacher did not believe me





8. Everyone just got into more trouble





9. My friends excluded me because I told the teacher





10. The teacher said it did not happen





11. The teacher really likes the student who was mean to me, so nothing was done about it





12. The teacher listened to me





Note: Frequency = the number of participants who answered that they had told the teacher that a child had been mean (n = 230) and subsequently reported that this “has happened to (me).” Children answered as many items as were relevant to their experience. Standardized loadings reported; all loadings are significant at the 5% level

Middle Years Development Instrument (Schonert-Reichl et al. 2013)

The Supportive Adults at School subscale of the Middle Years Development Instrument (MDI) was adapted from the School Support subscale of the California Healthy Kids Survey (CHKS; WestEd 2005) and measures children’s sense of connectedness with adults at school (Oberle et al. 2014). The statement “At my school there is a teacher or another adult,” preceded the items: “Who really cares about me,” “Who believes that I will be a success,” and “Who listens to me when I have something to say” (α=.71, n = 3026; grade 4, Canada) (Schonert-Reichl et al. 2013, p. 354). The subscale was measured on a 4-point scale, 0 (not at all true) to 3 (always true). A higher answer indicated higher perceived connectedness with adults in the school environment. It was expected that children who reported a higher level of adult connectedness would have a lower score on the experience factor and a higher score on the heard factor supporting convergent validity. This subscale of the MDI was included in phase 1 of our research and consistent with a recommendation that each scale contains a minimum of four items to identify an effective factor (Russell 2002); an additional item “Who I can talk to about my problems” was added. Results showed an equivalent internal consistency of the three and four item subscales: α = .888 (original three-item scale) and α = .889, CR = 0.896 (four-item scale). The four-item scale was used in phase 2.

Statistical Analyses

Phase 1

First, the fit of the items to a structure of student’s experience of teacher support was assessed. This was assessed using a method of exploratory factor analysis (EFA) in SPSS with Principal Axis Factoring (PAF) and promax rotation as recommended by Osborne and Costello (2009). Missing data were deleted listwise. A minimum Kaiser-Meyer-Olkin (KMO) of .60 suggested sufficient shared variability among items to produce a reliable factor (Beavers et al. 2013). The two-factor structure was confirmed by parallel analysis, scree plot, and eigenvalue criteria (≥1) (Osborne and Costello 2009). Second, confirmatory factor analysis (CFA) was conducted using MPlus statistical software using the WLSMV estimator to account for small sample sizes and binary variables (Byrne 2012; Muthén and Muthén 2015). Model fit was evaluated by normed chi-square value < 3 (Kline 2005), root-mean-square error of approximation (RMSEA < .08 or a 90% confidence interval that straddles .05) (Byrne 2012), comparative fit index (CFI > .90), and the Tucker-Lewis index (TLI > .90) (Marsh et al. 2011). Factor loadings above .50 were considered good and were accepted if they were above .30 (Osborne and Costello 2009). Communality (R2 > .40) supported the relatedness of an item to the other items in the factor (Osborne and Costello 2009). In the EFA, internal consistency of each scale was evaluated by Cronbach’s alpha (≥.70); in the CFA, the internal consistency was evaluated by the composite reliability index of Raykov (CR) (1997).

Phase 2

Data were initially assessed using SPSS for frequencies and missing data, distribution, and tolerance (values < .20 indicate multicollinearity) (Allen and Bennett 2010). First, CFA of the hypothesized two-factor model of the new instrument was conducted in MPlus using the WLSMV estimator to account for small sample size, binary variables, and non-normal distribution of data (Byrne 2012). There were no missing data; fit indices were reported as for phase 1. As recommended by Marsh et al. (2011), cross-loadings among subscales in measures of bullying appear to be a norm; items were therefore free to cross load onto other factors. Modification indices (MIs) were subsequently examined in order to identify the cross-loading of items and improve the fit of the CFA (Marsh et al. 2011). It was important to ensure that modifications to the model made sense based on the evidence-based literature (Muthén and Muthén 2015).

Third, the invariance of the factor structure across grades (grade 4 and combined grades 5 and 6 to account for lower student numbers in these grades at some participating schools), schools (school 2 and school 3), and gender (female, male) was assessed with multigroup analyses using robust maximum likelihood (MLM) to account for non-normal distribution of data (Byrne 2012). Consistent factor structure was implied by a corrected MLM chi-square (∆χ2) value that was non-significant (p > .05) (Byrne 2012). A configural model incorporated the baseline model for each group, multiple group analysis then tested for equivalence of factor loadings, residual covariance, and factor variance.

Fourth, construct validity of the instrument was assessed by reporting the correlation between the experience factor and the MDI Adults at School factor, and the heard factor and the MDI Adults at School factor, calculated in MPlus using the WLSMV estimator. Fifth, test-retest reliability was assessed using a subset of data from one school. The scores from each scale were summed in SPSS to create new variables experience test, experience retest, heard test, and heard retest. The assumption of normality and linearity was met for the experience factor but not the heard factor; the correlation was therefore reported by the Spearman’s rho (rS) (McCrae et al. 2011).


Phase 1

Sixty-nine students (62%) responded “yes” to the stem question “Have you ever told a teacher when another student has been mean to you or someone else on purpose?” These 69 students were then prompted to identify which items in a list of 13 items corresponded to the teacher’s subsequent behavior. The frequency of student responses to each item of the SETSS is shown in Table 1. As Table 1 shows, the most frequent response (76.8%) was to each item “It got solved,” and “The teacher helped me.” The least frequent response (20.3%) was to the item “The teacher thought it didn’t happen”.

A clear two-factor solution emerged in SPSS explaining 42.3% of the variance among all items. The KMO of .768 suggested sufficient variability among items to produce a reliable factor solution (Kaiser 1974, p. 35). Table 1 reports the loadings of items onto their respective factor (factor loadings .508 to .771). CFA of the 2-factor structure was run on the resulting 11-item scale. A model of good fit resulted (normed χ2 = 1.31, RMSEA = .067 [95%CI .00 to .111], CF1 = .966, TLI .957). Factor loadings were .625 to .885 for factor 1 (CR = .925) and .729 to .906 for factor 2 (CR = .885). The lowest communality of an item was found for item 1 “The teacher thought it was just an accident” (R2 = .39); the R2 of all other items ranged between .43 (item 6) and .82 (item 2). Factor 1 was named experience, and factor 2 was named heard (Table 1).

Phase 2

A total of 230 of 337 (68%) participants answered the SETSS having reported that they had told a teacher “that another student had been mean.” The frequency of response to each item is listed in Table 2. The most frequent response (81.3%) was to the item “The teacher listened to me,” and the least frequent response (13%) was to the item “The teacher said it didn’t happen.”

First, CFA revealed adequate fit of the hypothesized model (normed χ2 = 2.23, RMSEA = .073 [90%CI = .056 to .091], CFI = .913, TLI = .892). Factor loadings for the eight-item experience factor were between .528 and .861 (CR = .89), and for the four-item heard factor, between .643 and .864 (CR = .864) (see Table 2). Because the 90% CI for the RMSEA did not contain a value of .05, the modification indices were reviewed, revealing a regression of item 4, “The teacher helped me” on the experience factor and a cross-loading of the experience factor by item 4 (MI = 21.34). Further inspection of the MIs showed a covariance of item 9, “My friends excluded me because I told the teacher,” with item 4 (MI = 11.84).

Second, item 4 was allowed to covary with item 9 resulting in acceptable model fit (n = 230): normed χ2 = 2.07, RMSEA = .068 [90% CI = .05 to .086], CFI = .927, TLI = .907. Further review of the MI’s revealed a covariance of item 4 with item 1 “The teacher said it was just an accident” (MI = 11.75); this covariance was added to the model resulting in improved model fit (normed χ2 = 1.89, RMSEA = .062 [90% CI = .043 to .081], CFI = .940, TLI .923) (see Fig. 1). Figure 1 shows the factor loadings of each item onto the respective factor and the correlations of item 4 onto items 1 and 9. The theoretical plausibility of these modifications is addressed in the discussion.
Fig. 1

Modified factor structure of the SETSS. The curved arrow on the left represents the correlation between factors; the straight arrows represent the factor loadings of each item, e1 = item 1. The curved arrows on the right of the model represent the covariance of items

Third, invariance testing was conducted on the two-factor model that was confirmed by CFA in the first step of data analysis. The model identified in CFA was used because it displayed a good fit for each configural model that was used for invariance testing. Invariance testing showed consistent results over school grade 4 and grades 5 and 6, and over schools 2 and 3, and by gender (see Table 3). The configural model for invariance by school, however, did not include items 2 “It got solved” and 5 “The teacher said we should forget about it” because these items did not fit the model for school 2.
Table 3

Multigroup analysis of the SETSS


χ 2




No. FParms


Invariance constraints

χ2 SCF


Δ MLM χ2diff





Total group analysis


COVERT- CFA (n = 230)






.059 [.040 .078]

MLM estimator


Multiple group grade invariance (grade 4 and combined grades 5 and 6)


Grade 4 (n = 104)






.061 [.018 .092]


Grades 5 and 6 (n = 126)






.052 [.000 .081]


MG grade

  - M1






.056 [.030 .078]

Configural model



  - M2






.056 [.031 .077]

Factor loadings constrained equal






  - M3






.055 [.029 .075]

Invariance of the structural model.







Multiple group school invariance

Experience by e1 e3 e7 e8-e11; Heard by e4 e6 e12 - model fit school 2

School 2 (n = 61)






.067 [.000 .121]


School 3 (n = 106)






.043 [.000 .086]


MG school

  - M1






.049 [.000 .084]

Configural model



  - M2






.033 [.000 .072]







  - M3






.031 [.000 .070]

Invariance of the structural model







Multiple group gender invariance


Girl - EFA






.043 [.000 .077]

No constraints e12 on e11


Boy - EFA






.051 [.000 .083]

No constraints e4 on experience


Girl - CFA (n = 114)






.059 [.019 .089]


Boy - CFA (n = 115)






.063 [.028 .092]


MG gender

  - M1






.061 [.037 .082]

Configural model



  - M2






.061 [.038 .081]

Factor loadings constrained equal






  - M3






.065 [.044 .085]

Invariance of the structural model






Note: MG, multiple group; M1, model 1; χ2 indicates MLM chi-square value; df, degrees of freedom; CFI, comparative fit index; TLI, Tucker-Lewis index; No. FParms, number of free parameters; RMSEA, root-mean-square error of approximation (90% confidence interval); χ2SCF, scaling correction factor; CD, difference test scaling correction; Δ MLM, Satorra-Bentler scaled χ2 difference test; dfdiff, difference in degrees of freedom between the nested model and the comparison model; p (significant at 0.05). There were no residual invariances to constrain equal in the configural model, and no common residual covariances in M2. In the invariance model, the factor variances and covariances were constrained equal

Fourth, the experience factor had a significant and negative correlation with the Supportive Adults at School subscale of the MDI (correlation = − .224, p = .020) supporting convergent validity. In addition, a significant and positive correlation was found between the heard factor and the Supportive Adults at School subscale of the MDI (correlation = .385, p < .001) (normed χ2 = 1.84, RMSEA = .061 [90%CI .047 to .074], CF1 = .933, TLI .920) supporting concurrent validity. Test-retest reliability of the experience factor was strong (n = 31), rS = 0.733 (p < .001) and was demonstrated with the heard factor (n = 30), rS = 0.496 (p = .005) (Allen and Bennett 2010).


The purpose of this research was to explore the latent factor structure of the Student Experience of Teacher Support Scale (SETSS) and assess the reliability and construct validity of the new self-report instrument designed to measure children’s experience when they tell a teacher that a student has perpetrated aggression. Our research was premised on children’s report that aggressive behavior is harmful because it is often hidden from teachers and other adults (Cross et al. 2009). Furthermore, children report that they are not confident in telling a teacher or other adult about their experience of being victimized (Oliver and Candappa 2007).

The items included in the instrument evolved from a thematic analysis of focus group discussions with children (n = 22, ages 9 to 12) (Nelson et al. 2018). The two-factor model of a 12-item instrument was evaluated in phase 2 of the research where it showed strong factor loadings and composite reliability. The first factor represents children’s experience associated with telling the teacher and the second represents children’s feeling of being heard by the teacher. The experience category reflected a range of items that, based on children’s report in focus groups, could be perceived by children as a feeling of not being heard. An examination of the covariance between some items of the heard factor with the experience factor, however, revealed the multifaceted nature of those items. Being heard did not always result in a positive experience over the longer term. For example, friends subsequently excluded some children who felt “heard” by the teacher. Some children who reported an experience of being told by the teacher “it was just an accident” also reported that the teacher had helped them. The name “heard” was chosen because a prerequisite to a positive experience after telling the teacher was to feel heard. These issues are addressed following a discussion of the model fit.

The 90% confidence interval of the RMSEA of the two-factor model of the SETSS did not include the recommended value of .05, suggesting a limitation of model fit. The two-factor model did demonstrate reliability over time and a consistent factor structure by grade at school and across two different schools. There was, however, only weak invariance by gender. As expected, convergent validity was demonstrated between the SETSS and connectedness with adults at school; the correlations were however lower than anticipated. We will discuss the weak invariance of the experience factor by gender and the correlations between the new instrument and the MDI; first, however, we discuss the fit of item 4 “The teacher helped me” onto the two-factor model.

The two-factor model of the SETSS was explored in MPlus consistent with the method used by Marsh et al. (2011) and item 4 “The teacher helped me” was allowed to covary with item 9 “My friends excluded me because I told the teacher” resulting in acceptable model fit. A strong covariance of .776 suggested that children who reported that the teacher helped commonly reported that as a result of telling the teacher they were excluded by their friends. Similarly, 45% of students who sought help from an adult found that bullying persisted or got worse, while only 25% reported that their situation improved (Cross et al. 2009). At the ages of 8 to 12, as children move into adolescence, emotions are strongly reflected in their self-appraisal and the formation of identity (Pfeifer and Peake 2012). Persisting social isolation can result in low self-worth, self-blame, and poor mental and physical health (Laursen and Hartl 2013). The task of appropriately supporting preadolescent children’s social and emotional development is placing an increasing demand on teachers (Cunningham et al. 2016). Teachers have reported that bullying is becoming more complex and challenging to detect and deal with and that attempts to stop the behavior can actually make it worse (Cunningham et al. 2016). Cross et al. (2011) found that almost 70% of teachers wanted more training to address aggression that is intentionally hidden from teachers. The burden of this hidden form of aggression reported by children and teachers is a strong indication that greater knowledge and understanding of these complex interpersonal processes is required to inform future management strategies.

A second covariance of item 4 “The teacher helped me” with item 1 “The teacher said it was just an accident” was added to the model, resulting in a strong positive association between the two items and a good fit of the two-factor model (see Fig. 1). In the focus group discussions, children spoke of lies told to the teacher, including the lie that intentional aggression “was just an accident” (Nelson et al. 2018). It is not possible to determine if the reported child is lying or not. It may be that the aggression was intentional and purposeful, but it may be that the victim has misinterpreted the actions of the reported aggressor. Due to their past experience, some children are vulnerable; they learn to become defensive and overly sensitive to rejection (Chango et al. 2012). This vulnerability predisposes children to poor mental health outcomes associated with attribution bias and low self-worth (McDougall and Vaillancourt 2015). However, social support is protective as it mediates the automatic stress response in the brain that is linked to rejection sensitivity (Chango et al. 2012; Eisenberger et al. 2007). When adults help children who have been bullied to understand alternative scripts through which they can respond, it can lead to reduced stress and less sensitivity (Pepler et al. 2010). Thus, the covariance of the two items “The teacher helped me” with “The teacher said it was just an accident” might reflect the appropriate emotional support of teachers for children who have responded defensively to accidental hurt. This would explain the lower communality of item 1 (R2 .39).

The final modification indices that warrant discussion is the cross-loading of the student experience factor onto item 4 “The teacher helped me” that occurred for boys in the invariance analysis, but not for girls (see Table 3). Although the rates of relational aggression between girls and boys are similar, the peer group structure of girls is generally more intimate (Casper and Card 2017). Because of the status associated with belonging to the group, girls may feel that they must stay in the popular group despite being bullied (Nelson et al. 2018). Socially integrated bullies who are popular use social intimacy with peers as a resource by which they dominate and unite their peer group, increasing power over the victim and holding control over peers through fear (Kiefer and Wang 2016). For this reason, help from the teacher might seem more inaccessible to girls than it does to boys.

We found a positive and significant correlation between the heard factor and the Supportive Adults at School subscale of the MDI, although the correlation was in the moderate range (.385). This correlation might reflect that the measure from the MDI focused on all adults in the school whereas the heard factor focused only on teachers. The SETSS measures children’s own experience after reporting aggression to the teacher (reported by victims of aggression or by peers who had tried to support the victimized child). Items identified in the experience subscale of the SETSS had previously emerged through qualitative analysis as influencing children’s experience of power imbalance (Nelson et al. 2018). Power imbalance is the main differentiating feature between aggressive behavior and bullying (Rodkin et al. 2015). This supports our understanding that aggression that is covert, or hidden from the teacher, will increase the power imbalance experienced by children who are bullied.

Power imbalance is relational; it occurs through social interactions within the context of the group and in response to cultural norms (Thornberg 2018). This includes the culture of the school or classroom; for example, bullying victimization is minimized when classroom teachers demonstrate respect and care for children (Thornberg et al. 2018). However, the authors of recent qualitative studies have found that some teachers consider bullying to be influenced by individual characteristics of children, either those who are bullied or those who bully others (Rosen et al. 2017; Thornberg 2018). This can result in teachers failing to be aware of aggression that is perpetrated by children of high social status, or failing to support children who report bullying (Rosen et al. 2017). Veenstra et al. (2014, p. 1141) recommend that to develop successful interventions, it is necessary to understand how teachers respond to bullying “in the eyes of students” and how this affects bullying prevalence. The SETSS provides a measure to understand how teachers respond to bullying from the perspective of students. Each question was identified through qualitative research with students. We agree with recent recommendations for interventions to focus on educating teacher and community members on the serious effects of bullying, with the aim of increasing awareness and empathy and reducing the harm of bullying (Thornberg et al. 2018; VanZoeren and Weisz 2018). The SETSS may be used in conjunction with measures of bullying to investigate this further.

Limitations and strengths

Ideally, phase 2 would have contained a larger sample of participants. However, many schools had already committed to other research and their calendars did not have room for participation in this study. The active consent rate of 55% has been described as “typical” for research in large urban schools (Kiefer and Wang 2016, p. 61). Because participants answered the new instrument in response to self-report of having told a teacher about an aggressive incident, the instrument was completed by 230 participants, limiting the ability to test the invariance model across each school grade and each participating school because of a lack of power. Invariance testing did however give support for a consistent factor structure across two different schools, and between grade 4 and combined grades 5 and 6 (children move to secondary school at grade 7 in Western Australia).


The findings of this study support our understanding that covert aggression increases the power imbalance perceived by the victim by removing teacher support. We have demonstrated that the SETSS is a valid and reliable instrument that can be used in conjunction with other measures to estimate the degree to which students experience teacher support in the context of covert aggression and bullying. The SETSS resulted in two subscales, experience and feeling heard by the teacher. Factor analysis demonstrated a negative correlation between each subscale. A covariance between items suggested that as a result of telling the teacher, children were commonly excluded by friends. It is anticipated that the SETSS will be used in future studies so that its credentials as a useful tool can be consolidated. At the very least, we hope that this study will draw the attention of other researchers to the role that teachers play in covert bullying so that they, too, may seek to find new ways to measure this aspect of the student-teacher relationship. It is important to understand the needs of teachers and to equip teachers with knowledge and understanding of student behavior, as well as strategies to give students appropriate support (Garandeau et al. 2016). The eventual goal is to reduce the harm caused by bullying behavior in schools so as to maximize the health and wellbeing of all students.


Compliance with ethical standards

Ethics approval was obtained from the University Human Research Ethics Committee; and the study was performed in accordance with the ethical standards as laid down in the National Statement on Ethical Conduct in Human Research (The National Health and Medical Research Council, the Australian Research Council and the Australian Vice-Chancellors’ Committee 2007). The Student Experience of Teacher Support Scale (SETSS) is copyright to the authors. Please contact the authors for permission to use any parts of the article that include the SETSS.


  1. ACARA (2015). Guide to understanding ICSEA (Index of Community Socio-educational Advantage) values. Accessed 21 September 2016.
  2. Allen, P., & Bennett, K. (2010). PASW statistics by SPSS: A practical guide version 18. South Melbourne, Australia: Cengage Learning Australia Pty Limited.Google Scholar
  3. Beavers, A. S., Lounsbury, J. W., Richards, J. K., Huck, S. W., Skolits, G. J., & Esquivel, S. L. (2013). Practical considerations for using exploratory factor analysis in educational research. Practical Assessment, Research & Evaluation, 18(6), 1–13.Google Scholar
  4. Bovaird, J. A. (2010). Scales and surveys: some problems with measuring bullying behaviour. In S. R. Jimerson, S. M. Swearer, & D. L. Espelage (Eds.), Handbook of bullying in schools: an international perspective (pp. 277–292). New York: Routledge.Google Scholar
  5. Bradshaw, C. P., Sawyer, A. L., & O’Brennan, L. M. (2007). Bullying and peer victimization at school: perceptual differences between student’s and school staff. School Psychology Review, 36(3), 361–382.Google Scholar
  6. Bronfenbrenner, U. (2001). Developmental ecology through space and time: a future perspective. In P. Moen, G. H. Elder, & K. Luscher (Eds.), Examining lives in context: perspectives on the ecology of human development (pp. 619–647). Washington DC: American Psychological Association.Google Scholar
  7. Bronfenbrenner, U., & Morris, P. A. (2006). The bioecological model of human development. In W. Damon & R. M. Lerner (Eds.), Handbook of Child Psychology (6th ed., Vols. 1–12, Vol. 1: Theoretical models of human development, pp. 793–828). Hobolen: Wiley.Google Scholar
  8. Byrne, B. M. (2012). Structural equation modeling with MPlus: basic concepts, application, and programming. Great Britain: Routledge.Google Scholar
  9. Casper, D. M., & Card, N. A. (2017). Overt and relational victimization: a meta-analytic review of their overlap and associations with social-psychological adjustment. Child Development, 88(2), 466–483. Scholar
  10. Chango, J. M., McElhaney, K. B., Allen, J. P., Schad, M. M., & Marston, E. (2012). Relational stressors and depressive symptoms in late adolescence: rejection sensitivity as a vulnerability. Journal of Abnormal Child Psychology, 40(3), 369–379. Scholar
  11. Craig, W. M., Pepler, D., & Atlas, R. (2000). Observations of bullying in the playground and in the classroom. School Psychology International, 21(1), 22–36. Scholar
  12. Cross, D., Shaw, T., Hearn, L., Epstein, M., Monks, H., Lester, L., & Thomas, L. (2009). Australian Covert Bullying Prevalence Study (ACBPS). Child Health Promotion Research Centre: Edith Cowan University, Perth. Accessed 2 April 2013.
  13. Cross, D., Epstein, M., Hearn, L., Slee, P., Shaw, T., & Monks, H. (2011). National Safe Schools Framework: policy and practice to reduce bullying in Australian schools. International Journal of Behavioral Development, 35(5), 398–404. Scholar
  14. Cunningham, C. E., Rimas, H., Mielko, S., Mapp, C., Cunningham, L., Buchanan, D., Vaillancourt, T., Chen, Y., Deal, K., & Marcus, M. (2016). What limits the effectiveness of antibullying programs? A thematic analysis of the perspective of teachers. Journal of School Violence, 15(4), 460–482. Scholar
  15. Davey, C., Burke, T., & Shaw, C. (2010). Children’s participation in decision-making: a children’s views report. National Children’s Bureau.
  16. Day, C., Stobart, G., Sammons, P., & Kington, A. (2006). Variations in the work and lives of teachers: relative and relational effectiveness. Teachers and Teaching, 12(2), 169–192. Scholar
  17. De Laet, S., Colpin, H., Vervoort, E., Doumen, S., Van Leeuwen, K., Goossens, L., & Verschueren, K. (2015). Developmental trajectories of children’s behavioral engagement in late elementary school: Both teachers and peers matter. Developmental Psychology, 51(9), 1292–1306. Scholar
  18. Eisenberger, N. I., Taylor, S. E., Gable, S. L., Hilmert, C. J., & Lieberman, M. D. (2007). Neural pathways link social support to attenuated neuroendocrine stress responses. NeuroImage, 35(4), 1601–1612. Scholar
  19. Ertesvåg, S. K. (2016). Students who bully and their perceptions of teacher support and monitoring. British Educational Research Journal, 42(5), 826–850. Scholar
  20. Garandeau, C. F., Vartio, A., Poskiparta, E., & Salmivalli, C. (2016). School bullies’ intention to change behavior following teacher interventions: Effects of empathy arousal, condemning of bullying, and blaming of the perpetrator. Prevention Science, 17(8), 1034–1043. Scholar
  21. Gest, S. D., & Rodkin, P. C. (2011). Teaching practices and elementary classroom peer ecologies. Journal of Applied Developmental Psychology, 32(5), 288–296. Scholar
  22. Hawley, P. H. (2003). Prosocial and coercive configurations of resource control in early adolescence: a case for the well-adapted Machiavellian. Merrill-Palmer Quarterly, 49(3), 279–309. Scholar
  23. Houghton, S. J., Nathan, E., & Taylor, M. (2012). To bully or not to bully, that is not the question: Western Australian early adolescents’ in search of a reputation. Journal of Adolescent Research, 27(4), 498–522. Scholar
  24. Kaiser, H. F. (1974). An index of factorial simplicity. Psychometrika, 39(1), 31–36.Google Scholar
  25. Kiefer, S. M., & Wang, J. H. (2016). Associations of coolness and social goals with aggression and engagement during adolescence. Journal of Applied Developmental Psychology, 44, 52–62. Scholar
  26. Kline, R. B. (2005). Principles and practice of structural equation modeling. New York: The Guilford Press.Google Scholar
  27. Laursen, B., & Hartl, A. C. (2013). Understanding loneliness during adolescence: developmental changes that increase the risk of perceived social isolation. Journal of Adolescence, 36(6), 1261–1268. Scholar
  28. Marsh, H. W., Nagengast, B., Morin, A. J. S., Parada, R. H., Craven, R. G., & Hamilton, L. R. (2011). Construct validity of the multidimensional structure of bullying and victimization: an application of exploratory structural equation modeling. Journal of Educational Psychology, 103(3), 701–732. Scholar
  29. McCrae, R. R., Kurtz, J. E., Yamagata, S., & Terracciano, A. (2011). Internal consistency, retest reliability, and their implications for personality scale validity. Personality and Social Psychology Review, 15(1), 28–50. Scholar
  30. McDougall, P., & Vaillancourt, T. (2015). Long-term adult outcomes of peer victimization in childhood and adolescence: pathways to adjustment and maladjustment. American Psychologist, 70(4), 300–310. Scholar
  31. Mishna, F., Pepler, D., & Wiener, J. (2006). Factors associated with perceptions and responses to bullying situations by children, parents, teachers, and principals. Victims & Offenders, 1(3), 255–288. Scholar
  32. Mucherah, W., Finch, H., White, T., & Thomas, K. (2018). The relationship of school climate, teacher defending and friends on students’ perceptions of bullying in high school. Journal of Adolescence, 62, 128–139. Scholar
  33. Muthén, L. K., & Muthén, B. O. (1998-2015). MPlus user’s guide. Seventh Edition. Los Angeles: Muthén & Muthén.Google Scholar
  34. Nelson, H. J., Kendall, G. E., Burns, S. K., & Schonert-Reichl, K. A. (2015). A protocol for the design of an instrument to measure preadolescent children’s self-report of covert aggression and bullying. BMJ Open, 5(e009084), 10. Scholar
  35. Nelson, H. J., Burns, S. K., Kendall, G. E., & Schonert-Reichl, K. A. (2018). The factors that influence and protect against power imbalance in covert aggression and bullying among preadolescent children: a thematic analysis. Journal of School Nursing, 34(4), 281–291. Scholar
  36. Norwalk, K. E., Hamm, J. V., Farmer, T. W., & Barnes, K. L. (2016). Improving the school context of early adolescence through teacher attunement to victimization effects on school belonging. The Journal of Early Adolescence, 36(7), 989–1009. Scholar
  37. Oberle, E., Schonert-Reichl, K. A., Guhn, M., Zumbo, B. D., & Hertzman, C. (2014). The role of supportive adults in promoting positive development in middle childhood: a population-based study. Canadian Journal of School Psychology, 29(4), 296–316. Scholar
  38. Oliver, C., & Candappa, M. (2007). Bullying and the politics of ‘telling’. Oxford Review of Education, 33(1), 71–86. Scholar
  39. Osborne, J. W., & Costello, A. B. (2009). Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Pan-Pacific Management Review, 12(2), 131–146.Google Scholar
  40. Pepler, D. J., Craig, W. M., & O’Connell, P. (2010). Peer processes in bullying: informing prevention and intervention strategies. In S. M. Swearer, S. R. Jimerson, & D. L. Espelage (Eds.), Handbook of bullying in schools: an international perspective (pp. 469–480). New York: Routledge.Google Scholar
  41. Pfeifer, J. H., & Peake, S. J. (2012). Self-development: Integrating cognitive, socioemotional, and neuroimaging perspectives. Developmental Cognitive Neuroscience, 2(1), 55–69. Scholar
  42. Polit, D. F., & Beck, C. T. (2006). The content validity index: are you sure you know what’s being reported? Critique and recommendations. Research in Nursing & Health, 29(5), 489–497. Scholar
  43. Raykov, T. (1997). Estimation of composite reliability for congeneric measures. Applied Psychological Measurement, 21(2), 173–184. Scholar
  44. Rodkin, P. C., Espelage, D. L., & Hanish, L. D. (2015). A relational framework for understanding bullying: developmental antecedents and outcomes. American Psychologist, 70(4), 311–321. Scholar
  45. Rosen, L. H., Scott, S. R., & DeOrnellas, K. (2017). Teachers’ perceptions of bullying: a focus group approach. Journal of School Violence, 16(1), 119–139. Scholar
  46. Russell, D. W. (2002). In search of underlying dimensions: the use (and abuse) of factor analysis. Personality and Social Psychology Bulletin, 28(12), 1629–1646. Scholar
  47. Schonert-Reichl, K. A., Guhn, M., Gadermann, A. M., Hymel, S., Sweiss, L., & Hertzman, C. (2013). Development and validation of the middle years development instrument (MDI): assessing children’s well-being and assets across multiple contexts. Social Indicators Research, 114(2), 345–369. Scholar
  48. Serdiouk, M., Berry, D., & Gest, S. D. (2016). Teacher-child relationships and friendships and peer victimization across the school year. Journal of Applied Developmental Psychology, 46, 63–72. Scholar
  49. Swan, P., & Riley, P. (2015). Social connection: empathy and mentalization for teachers. Pastoral Care in Education, 33(4), 220–233. Scholar
  50. Swift, L. E., Hubbard, J. A., Bookhout, M. K., Grassetti, S. N., Smith, M. A., & Morrow, M. T. (2017). Teacher factors contributing to dosage of the KiVa anti-bullying program. Journal of School Psychology, 65, 102–115. Scholar
  51. The National Health and Medical Research Council, the Australian Research Council and the Australian Vice-Chancellors’ Committee (2007). National statement on ethical conduct in human research. Commonwealth of Australia, Canberra. Accessed 18 July 2014.
  52. Thornberg, R. (2018). School bullying and fitting into the peer landscape: a grounded theory field study. British Journal of Sociology of Education, 39(1), 144–158. Scholar
  53. Thornberg, R., Wänström, L., Pozzoli, T., & Gini, G. (2018). Victim prevalence in bullying and its association with teacher–student and student–student relationships and class moral disengagement: a class-level path analysis. Research Papers in Education, 33(3), 320–335. Scholar
  54. UNICEF (1989). Convention on the rights of the child. Accessed 6 April 2017.
  55. Vaillancourt, T., & Hymel, S. (2006). Aggression and social status: the moderating roles of sex and peer-valued characteristics. Aggressive Behavior, 32(4), 396–408. Scholar
  56. VanZoeren, S., & Weisz, A. N. (2018). Teachers’ perceived likelihood of intervening in bullying situations: individual characteristics and institutional environments. Journal of School Violence, 17(2), 258–269. Scholar
  57. Veenstra, R., Lindenberg, S., Huitsing, G., Sainio, M., & Salmivalli, C. (2014). The role of teachers in bullying: the relation between antibullying attitudes, efficacy, and efforts to reduce bullying. Journal of Educational Psychology, 106(4), 1135–1143. Scholar
  58. Volk, A. A., Veenstra, R., & Espelage, D. L. (2017). So you want to study bullying? Recommendations to enhance the validity, transparency, and compatibility of bullying research. Aggression and Violent Behavior, 36, 34–43. Scholar
  59. WestEd (2005). California healthy kids survey: middle school survey, Module A.

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.School of Nursing, Midwifery and ParamedicineCurtin UniversityPerthAustralia
  2. 2.School of Public Health and Collaboration for Evidence, Research and Impact in Public HealthCurtin UniversityPerthAustralia
  3. 3.Educational and Counselling Psychology, and Special EducationUniversity of British ColumbiaVancouverCanada
  4. 4.Human Early Learning Partnership, School of Population and Public Health, Faculty of MedicineUniversity of British ColumbiaVancouverCanada
  5. 5.School of PsychologyCurtin UniversityPerthAustralia

Personalised recommendations