Background

Over the past two decades, there has been a substantial increase in resilience research [1, 2], following dissatisfaction with ‘deficit’ models of illness and psychopathology [3]. Resilience is defined as the maintenance of positive adjustment in the context of exposure to significant adversity [4]. Key protective factors that confer resilience include positive individual characteristics, functional family relationships and a supportive environment outside the family [5, 6]. Individual characteristics such as self-control, empathy, intelligence, self-esteem and problem-solving skills have been identified as beneficial whether someone is facing low or high adversity [e.g. 7]. Similarly, warm relationships within the family and well-structured home environments are important for positive development in all children even in the absence of exposure to stressful life events. However, having a supportive family has been shown to be particularly important for children trying to cope with stressful experiences [e.g. 8]. Lastly, supportive environments outside the family such as availability of social support, school connectedness, having good neighbours and positive role models have been identified as potential protective factors [e.g. 5].

While resilience is conceived of as an end product of buffering processes that do not eliminate risks and stress but allow the individual to deal with them effectively, protective factors have been viewed as moderators of risk and adversity that enhance positive (i.e. developmentally appropriate) outcomes [4, 9]. The measurement of a range of factors that promote positive outcomes is crucial to explore the mechanisms between protective factors and risk factors, and to design intervention and prevention strategies. The student resilience survey [SRS; 10] covers a range of external supports and internal characteristics which can potentially be viewed as protective factors. It was constructed by combining elements from two surveys: the California Healthy Kids Survey [11] and the Perceptions of Peer Support Scale [12], assessing student perceptions of their individual characteristics, protective resources from family, peer, school and community.

The initial SRS development study by Sun and Stewart has supported the validity of the scale [10]. However, there were several limitations to the SRS validation. Firstly, the samples of children were drawn from only 20 primary schools in the state of Queensland, Australia. Sampling from a larger selection of schools, and a wider geographical area, is clearly needed for a more robust assessment of the psychometric properties of this measure. Secondly, although the initial validation study comprised a large sample size (n = 2794), this study will further provide confirmation by including reports from over 7000 children. Thirdly, SRS has only been validated using a scale-level approach. Scale-level analysis does not account for how individuals at different levels of the latent construct perform on the individual items of an instrument [13]. Item-level approaches allow examination of how individual subject responses on items of an instrument relate to an unobservable trait [13]. Differential item functioning [DIF; 14] allows for investigating item response probability based on different groups. DIF is present when individuals from different sociodemographic groupings (such as gender or ethnicity) have a different probability of answering an item [15]. Lastly, the initial validation did not investigate the association between SRS subscales and mental health outcomes. It is expected that most of the SRS subscale scores will be negatively correlated with emotional and behavioural problems [e.g. 16, 17] and attainment of good health [18].

The SRS can be an important tool in assessing the impact of protective factors when investigating the relationship between risk and psychological outcome and development. The purposes of the present study were threefold. First, we aimed to replicate the psychometric characteristics of the SRS found in its initial investigation, this time with an English sample [10]. Second, we aimed to investigate the measurement invariance in regard to several subgroups. Third, we aimed to assess the relationships between the SRS domains and children’s mental health outcomes.

Methods

Sample

Data were collected in 2015 from children who were part of a large project that focused on the promotion of resilience and emotional wellbeing (‘HeadStart’, funded by the Big Lottery Fund) in 12 local areas across 90 schools, England. The analyses reported are based on surveys completed by 7663 pupils (42.3% male); 1967 pupils were in primary school (year 6, mean age = 11.38, SD = 0.29) and 5696 pupils were in secondary school (years 7, 8 and 9, mean age = 13.31, SD = 0.86). For the item-level DIF analysis all items needed to be complete, hence only pupils who completed all items were included (sample size ranged from 6047 to 6123).

The sample was not drawn to be representative of all school children in England; it was based on local areas that were part of the HeadStart programme and each of the 12 local areas selected the schools to participate [19]. Overall, 5496 (72.8%) of pupils were White British (compared to the national average of 76.2%), and 6176 (81.6%) pupils’ first language was English (compared to the national average of 82.5%). 1452 (19.1%) were eligible for free school meals (compared to the national average of 16.2%—including nursery schools), 131 (1.7%) had a statement of special educational needsFootnote 1 (compared to the national average of 2.8%) and a further 1159 (15.1%) had any elevated special educational needs, albeit not great enough to meet the threshold for a full statement of SEN.

Measures

Student resilience survey (SRS)

The SRS is a 47-item measure comprising 12 subscales measuring students’ perceptions of their individual characteristics as well as protective factors embedded in the environment. Frequency of each item was rated on a 5-point scale (1 = never to 5 = always).

As this was part of a larger project and it was important not to burden the pupils with a long survey, only 10 of the SRS subscales (out of 12) were selected to be included into the survey with other validated measures. The 10 chosen subscales were: family connection, school connection, community connection, participation in home and school life, participation in community life, peer support, self-esteem, empathy, problem solving, and goals and aspirations. As the main project was interested in identifying protective factors in a child’s life, the pro-social peers (two items: my friends try to do what is right; my friends do well in school) and communication and cooperation (three items: I help other people; I enjoy working with other students; I stand up for myself) subscales were not included. Moreover, the scale was adapted to English school children based on discussions with young advisors from Common Room (a young people’s advocacy and engagement group with a specific focus on disability, health and mental health). Four items were edited so that they were more general and suitable for school-aged children in England (i.e. instead of “are there students at your school who would ask you to play when you are all alone”, it has been changed to “are there students at your school who would ask you to join in when you are all alone”; instead of “are there students at your school who would help you if you hurt yourself in the playground”, it has been changed to “are there students at your school who would help you if you hurt yourself”; instead of “are there students at your school who would invite you to play at their home”, it has been changed to “are there students at your school who would invite you to their home”; instead of “are there students at your school who would share things like stickers, toys & games with you”, it has been changed to are there students at your school who would share things with you”). Lastly, one item from the peer support scale (tell you you’re good at things) was omitted.

Mental health difficulties were measured with the me and my feelings questionnaire (formerly known as Me and My School measure, M&MS). It is a 16-item measure comprising a 10-item emotional difficulties scale and a 6-item behavioural difficulties scale [20, 21]. Each item includes a short statement (e.g. I am lonely; I get angry) measured on a 3-point Likert scale (0 = never, 1 = sometimes, and 2 = always) (emotional problems sum score mean = 5.17, SD = 3.87; behavioural problems sum score mean = 3.05, SD = 2.52). Cronbach’s αs in the current sample were 0.84 for emotional problems (n = 7187) and 0.80 for behavioural problems (n = 7243).

Global subjective distress was measured with child outcome rating scale (CORS). CORS consists of four items: how am I doing; how are things in my family; how am I doing at school; and how is everything going. The rating scale is a 10 cm line with a happy face at one end and a sad face at the other; children are asked to put a mark on the line to indicate the place that best describes how they feel. The score for each item is automatically recorded and the overall score can range from 0 to 40 (sum score mean = 9.59, SD = 7.7); higher scores indicate more global subjective distress [22]. Cronbach’s α in the current sample was 0.81 (n = 7448).

Impact of health on daily life was measured with the EQ 5D-Y [23]. It has five dimensions: mobility (‘walking about’), self-care (‘looking after myself’), usual activities (‘doing usual activities’), pain and discomfort (‘having pain or discomfort’) and anxiety and depression (‘feeling worried, sad or unhappy’). All items refer to the health state ‘today’. Each item has three levels of problems reported (1 = no problems, 2 = some problems and 3 = a lot of problems) (sum score mean = 6.20, SD = 1.46). Cronbach’s α in the current sample was 0.65 (n = 7038).

Health today was also measured using the EQ 5D-Y. It included a visual analogue scale where the children rated their overall health status on a scale from 0 to 100 with 0 representing the worst and 100 representing the best health state they can imagine (on that day). In the current study, it was recoded so that higher scores indicated worse health (sum score mean = 20.64, SD = 19.8).

Special educational needs (SEN), eligibility for free school meals (FSM), and English as an additional language (EAL) were derived from the national pupil database (NPD). SEN were based on the school’s assignment of a child to a level of special educational needs. Children with SEN, whether with or without statement, were considered as having special educational needs. FSM is frequently used as an indicator of low family income since only families on income support are entitled to claim free school meals. Lastly, EAL was coded as present if a child’s first language was not English.

Procedure

Ethical approval was obtained from the University College London Research Ethics Committee. Children completed questionnaires using a secure online system during their usual school day with parent consent. Before pupils responded to the survey, teachers read an information sheet to them which highlighted confidentiality of their answers as well as their right to withdraw from the study. Children provided informed consent prior to completing the survey. The online system was designed to be easy to read and child friendly.

Analyses

The structure and psychometric properties of the SRS were investigated in several stages. Firstly, confirmatory factor analysis (CFA) was conducted, using Mplus version 7.11 [24], to confirm whether constructs identified as subscales in previous research of this measure are evident in the current sample. This analysis was controlled for intra-class correlation due to clustering by schools [25]. Secondly, differential item functioning (DIF) was investigated across a range of demographic groupings using the Mantel–Haenszel procedure and the Liu–Agresti common log odds ratio as a measure of effect size [26] in DIFAS 5.0 [27]. Thirdly, DTF was conducted to examine the measurement invariance directly at the scale level across different subgroups in DIFAS 5.0. Fourthly, Cronbach’s α and McDonald’s ω were calculated, using SPSS version 21 and R, to assess the reliability of the subscales. Fifthly, to identify the association between protective factors and mental health outcomes, correlations were run between the SRS subscales and mental health outcomes using SPSS version 21. Lastly, to investigate whether internal or external factors had an impact on mental health outcomes, all subscales of the SRS were entered into regression models at the same time predicting each of the health outcomes. Both unadjusted and adjusted (adjusted for gender, school level—primary/secondary—SEN, EAL and FSM) random effects linear regression analyses (allowing for different school intercepts) were run using STATA version 12; unstandardized Bs, standard error and p-values are reported.

Results

Factor structure

Confirmatory factor analysis for ordinal data with weighted least squares with robust standard errors, mean, and variance adjusted (WLSMV) estimator [28] was carried out by testing a model with 10 correlated factors indicated by previous research (Table 1). Given the large sample size, Chi-square was not used to test model fit [29]. Other fit indices (CFI = 0.99; TLI = 0.99; RMSEA = 0.01, SRMR within = 0.03; n = 7663) indicated a good model fit based on widely accepted criteria [30]. The correlation between 10 latent factors ranged between 0.26 and 0.77 (Table 2).

Table 1 CFA standardised loadings, measurement errors and intra-class correlations (by school)
Full size table
Table 2 Factor correlations
Full size table

Reliability

Since Cronbach’s α as a single measure for reliability is no longer regarded as optimal [31], McDonald’s ω was also used. Coefficient ω gives a better estimate of reliability than Cronbach’s α if the items of a scale are not tau equivalent [32, 33]. McDonald’s ωs were determined using the factor loadings of the multilevel confirmatory factor analysis (within-school factor models; only for the subscales with more than 2 items). The internal consistency for all the subscales was good. Cronbach’s α was 0.80 and McDonald’s ω was 0.89 (n = 7360) for the family connection subscale; α was 0.89 and ω was 0.91 (n = 7332) for the school connection subscale; α was 0.91 and ω was 0.94 (n = 7286) for the community connection subscale; α was 0.79 and ω was 0.84 (n = 7288) for the participation in home and school life subscale; α was 0.74 (n = 7304) for the participation in community life subscale; α was 0.80 and ω was 0.85 (n = 7358) for the self-esteem subscale; α was 0.77 (n = 7391) for the empathy subscale; α was 0.83 and ω was 0.87 (n = 7314) for the problem-solving subscale; α was 0.73 (n = 7324) for the goals and aspirations subscale; lastly α was 0.93 and ω was 0.96 (n = 7052) for the peer support subscale.

Differential item functioning (DIF) and differential test functioning (DTF)

In order to examine whether items behaved equivalently across a range of different subgroups of children, DIF analyses were undertaken for all subscales with more than two items. The non-parametric Mantel–Haenszel procedure was chosen to test for DIF since it is not based on the assumptions of a specific item response model [34]. Nevertheless, the subscales were checked to be sufficiently unidimensional based on a single factor multi-level CFA and which was acceptable according to standard criteria for all subscales and only mild violations for ‘participation in home and school life’ were found (see Additional file 1: Table S1 for details). Further, whether the CFA model’s thresholds were ordered along the latent continuum was inspected. Higher item categories corresponded to higher trait levels and only the space on the latent trait corresponding to category 2 was for some items comparatively small [35] (see Additional file 2: Table S2 for item thresholds).

In the DIF analysis, six grouping criteria were examined: gender, primary/secondary school level, whether the child had any elevated special educational need (SEN), whether English was the child’s second language (EAL), and whether the child was eligible for free school meals (FSM). Boys (42.3%, n = 2591), secondary school children (75.0%, n = 4592), children with SEN with or without statement (18.6%, n = 950), non-native English speakers (17.6%, n = 1064), and children receiving FSM (18.4%, n = 1116) were the focus of these investigations (and formed the focal group in DIF analyses). DIF analyses compare the item endorsement rates in the focal group compared to reference group (e.g. children with SEN with or without statement compared to all other children), conditioning on test scores. If children with the same overall score on a subscale (e.g. overall family connection subscale score) have different probabilities of endorsing an individual item then this item is said to show differential item functioning; in other words the item behaves differently across the two groups.

The Liu-Agresti common logs ratio [L-A-LOR; 26] was used to assess the size of potential DIF effects and to gauge their potential relevance. Positive L-A-LOR values indicate the item is more difficult to endorse for the focal group, while negative L-A-LOR values indicate that the item is more difficult to endorse for the reference group, given the same level of underlying trait. The magnitude of DIF was interpreted using a widely accepted classifying system [36]: the magnitude was negligible if L-A-LOR was less than 0.43, moderate if L-A-LOR was between 0.43 and 0.64, and large if L-A-LOR was greater than 0.64.

According to the DIF analysis (Table 3), boys were more likely to agree that students at school were more likely to pick them for a partner and girls were more likely to agree that students told them their secrets and missed them if they weren’t at school. Primary school children were more likely to agree that they do things at school which make a difference; that their peers explain to them the rules if they don’t understand, and that their peers help them if they hurt themselves. Secondary school children were more likely than primary school children to endorse that there are students at their school who would tell them their secrets. Children with SEN were more likely to agree that their family listens to them when they have something to say. Lastly, children without EAL were more likely to agree that peers invite them to their home. Analyses did not indicate any DIF based FSM eligibility.

Table 3 Differential item functioning and differential test functioning for the resilience scale
Full size table

Differential test functioning (DTF) assesses the aggregate effect of DIF across all the items in a scale [37]. A scale with a DIF effect variance of ν2 below 0.07 can be classified as having small DTF, whereas DTF would be considered medium for 0.07 ≤ ν2 ≤ 0.14 and large for ν2 > 0.14 [37]. All subscales, except the peer support subscale, showed small DTF whereas the peer support subscale showed moderate DTF (Table 3).

The student resilience survey and mental health problems

A pattern was identified whereby the subscales of the SRS are negatively associated with mental health difficulties, global subjective distress and impact on health (Table 4). In terms of negligible associations, empathy demonstrated very low correlations with both emotional problems and health. On the other hand, both self-esteem and problem solving had moderate correlations with emotional problems and global subjective distress.

Table 4 Correlations between student resilience subscales and other scales
Full size table

Furthermore, in order to investigate whether internal or external factors had an impact on mental health outcomes, all subscales of the SRS were entered into linear regression models (with school as a random effects term) at the same time predicting each of the health outcomes (Table 5). The adjusted regression results showed that family connection, self-esteem, problem solving and peer support were negatively associated with all the mental health outcomes. On the other hand, those who had high empathy were more likely to display mental health difficulties, global subjective distress and impact on health.

Table 5 Unadjusted and adjusted associations between student resilience survey subscales and other scales
Full size table

Discussion

The aim of this study was to replicate the psychometric properties of the student resilience survey (SRS) within an English sample, to investigate the measurement invariance in subgroups, and to investigate the relationship between SRS subscales and mental health.

On the basis of the confirmatory factor analysis, the factor structure of the measure was similar to the original validation study showing that 41 items loaded uniquely onto their respective 10 subscales. Moreover, similar to the validation study, the analyses provided evidence for acceptable reliability of all the subscales for this sample, especially considering that some of them are very short//have very few items//something like that [10]. Nevertheless, the ‘participation in home and school life’ showed in individual analyses that it is not unidimensional and will need further research investigating its structure.

This study extended previous research by generating evidence of the SRS’s validity via analysis of DIF and DTF. Our DIF analysis indicated differences between boys and girls in terms of how they answered the peer support subscale of the SRS. This may be due to the differences between boys’ and girls’ expectations from friends. One study has shown that boys describe friends as “people you play with” whereas girls describe them as “people you can trust” [38]. Moreover, DIF analyses suggested that children in secondary school were more likely to agree that their peers shared secrets with them. This is in line with literature suggesting that during middle childhood, the quality of friendship changes from relationships characterised by the temporary sharing of activities and leisure time to friendships enhanced by intimacy, help, certainty, loyalty and confidence [39]. Primary school children were also more likely to agree that their peers explained the rules of play to them and helped if they hurt themselves. This may be due to children in secondary school becoming more independent and working through their problems more easily than younger children [40]. Children with special educational needs were more likely to agree that their families listened to them when they have something to say. The quality of caregiving by parents is crucial for children with special educational needs [41], hence parents of children with special educational needs might be making an extra effort to engage with their children. Children with English as a first language were more likely to agree that peers invited them to their home. Literature suggests that children prefer same-ethnic friends, even when the opportunity to meet out-group friends has been taken into account [42]. Hence, children with English as a first language might get more invites to friends’ homes. All subscales, except the peer support subscale, showed small DTF whereas the peer support subscale showed moderate DTF. Future research should explore in more detail whether the DIF found is due to qualitative differences and constitutes relevant item and scale bias that needs to be addressed by changing items or separate norming tables (especially for school and gender [43]).

As expected on the basis of related evidence, our results showed negative correlations between all the SRS subscales and emotional, behavioural and health problems. Adjusted random effects linear regression models showed that family connection, self-esteem, problem solving and peer support were negatively associated with emotional, behavioural and health problems. The capacity of supportive families to buffer children from the impact of stressful life events is well documented [44]. For example, family-related factors such as parenting style and parent–child relationship quality have been associated with improved psychosocial adjustment [e.g. 45, 46] and attainment of good health [18]. Having a positive parent–child relationship may translate into an opportunity for parents to guide their children in understanding emotions and dealing with stress. Additionally, it has been shown that healthy peer relationships and healthy school environment can protect youth from maladjustment (i.e. emotional and behavioural problems) [47] and health problems [18]. Similar to previous literature, self-esteem and problem-solving skills have been negatively associated with emotional problems [48, 49]. Problem-solving skills and self-esteem may enable children to alter the source of stress which in return may have a positive impact on mental and physical health problems [50]. On the other hand our results showed that the correlation between empathy and emotional problems was very low. Moreover, when all the subscales were loaded into the random effects regression models predicting mental health outcomes, empathy was positively associated with emotional, behavioural and health problems. The literature suggests that extreme empathy may increase vulnerability to mental health difficulties [51, 52].

In general, protective factors alter responses to adverse events so that potential negative outcomes can be avoided [9]. According to Benzies and Mychasiuk [53], resilience is optimised when protective factors are strengthened at all interactive levels of the socio-ecological model (i.e. individual, family and community). Identifying protective factors that promote resilience to mental health problems could lead to improved intervention strategies [1].

It is important to note the methodological limitations of the study. Firstly, the population of the study was not drawn to be representative of all school children in England; however, the participants were from 12 local areas of England which increased the generalisability of the results. Secondly, due to the overlap between the items of the SRS and the mental health difficulties questionnaires and the cross-sectional nature of this study, we cannot disentangle temporal precedence (what comes first) and causal mechanisms between the resilience factors and mental health outcomes. However, future studies should apply longitudinal methods to elucidate the relationship between the SRS protective factors and maladjustment. Thirdly, this study did not investigate the stability of scores; hence future studies should examine the test–retest reliability of the SRS. Lastly, our approach addressed the clustering due to schools in our sample with an appropriate multilevel approach to identify between-student variation [25]; but we did not model the differences between schools. The corresponding SRMR values for the between-school variance–covariance matrices were consequently bad (SRMR = 0.61 for full model; see also Additional file 1: Table S1). The intra-class correlations presented in Table 1 also show that relevant amounts of variance in item responses were due to differences between schools. Future research should explore school-characteristics to further explore these differences. Additionally, future research may investigate whether the SRS could be effectively or appropriately utilised with children who have clinical levels of mental health problems.

Conclusions

To the best of our knowledge, this is the first study to investigate the validity of the student resilience survey in the UK and to explore its relationship with mental health and wellbeing. Notwithstanding the limitations, our results showed that SRS subscales have loaded uniquely onto their respective 10-factor structure. All the SRS subscales had high reliability and the subscales (except for empathy) were negatively associated with mental health problems, global subjective distress and impact on health. Given the interest of investigating the relationship between risk, psychological outcomes and development, the SRS provides an exciting possibility to assess several different protective factors, and can thus be used as a valuable measurement tool in resilience and risk factor research.