Online Pestkoppenstoppen: systematic and theory-based development of a web-based tailored intervention for adolescent cyberbully victims to combat and prevent cyberbullying
- 3.7k Downloads
The purpose of this article is to give an integrative insight into the theoretical and empirical-based development of the Online Pestkoppenstoppen (Stop Bullies Online/Stop Online Bullies). This intervention aims to reduce the number of cyberbully victims and their symptoms of depression and anxiety (program goal), by teaching cyberbully victims how to cope in an adequate and effective manner with cyberbully incidents (program’s outcomes).
In developing the program the different steps of the Intervention Mapping protocol are systematically used. In this article we describe each step of Intervention Mapping. Sources used for the development were a literature review, a Delphi study among experts, focus group interviews with the target group, and elements from a proven effective anti-bullying program. The result is a fully automated web-based tailored intervention for cyberbully victims (12-15 years) consisting of three web-based advice sessions delivered over three months. The first advice aims to teach participants how behavior is influenced by the thoughts they have, how to recognize and dispute irrational thoughts and how to form rational thoughts. In the second advice, participants will learn about the way bullying emerges, how their behavior influences bullying and how they can use effective coping strategies in order to stop (online) bullying. In the third advice, participants receive feedback and will learn how to use the Internet and mobile phones in a safe manner. Each advice is tailored to the participant’s personal characteristics (e.g., personality, self-efficacy, coping strategies used and (ir)rational thoughts). To ensure implementation of the program after testing it for effectiveness, the intervention was pretested in the target-population and an implementation plan was designed. Finally, we will elaborate on the planned randomized controlled trial in which the intervention will be compared to a general information group and waiting list control group. This evaluation will provide insight into the intervention’s efficacy to reduce cyberbullying and its negative effects.
Intervention Mapping is a time consuming but profound way to ensure that each step of developing an intervention is taken, and resulted in three web-based tailored pieces of advices that teach adolescents how to cope more effectively with cyberbullying experiences.
Keywords(Cyber) bullying Intervention Web-based E-health Computer tailoring Behavior change Intervention Mapping
Pleasure at school
Rational Emotive (Behavioral) Therapy
Active Problem Solving
Change Objectives DT, Dynamic Tailoring
Randomized Control Trial
The growing popularity of social media (e.g. Facebook, Twitter) and instant messenger services (e.g. IMchat, Twitter)  does not always lead to positive experiences. Cyberbullying is defined as a repeated aggressive and intentional act, carried out by a group or an individual, using electronic forms of contact. This act is directed towards a victim who cannot easily defend him or herself . Between 20% and 40% of the adolescents worldwide report being a cyberbully victim . Research has indicated that cybervictimization is associated with serious internalizing difficulties such as depression [4, 5], anxiety , emotional distress  and suicidality [7, 8]. Cyberbully victims also more often have experienced drugs, alcohol, physical or sexual abuse, have displayed delinquent and aggressive behavior, have problems at school and have dropped out of school [9, 10, 11, 12, 13, 14, 15, 16]. The more adolescents engage in cyber aggression, the more loneliness they feel, the lower their global self-worth is, the fewer mutual friendships they have and the lower their ratings of social acceptance and popularity by peers are . Cyberbullies as well as cyberbully/victims (i.e. both victim and bully) have worse subjective health than those who are not involved , and victims of both traditional and cyberbullying are four times more likely to experience depressive symptoms and are five times more likely to attempt suicide compared to non-victims .
Unfortunately, intervention programs that deal specifically with cyberbullying are scarce . Moreover, students perceive that methods used in traditional bullying incidents are not equally effective in cyberbullying incidents . Therefore, there is an urgent need for effective cyberbullying interventions. Ideally, these interventions should: (1) do more than increase awareness of potential threats of the Internet by offering victims intensive intervention strategies on the basis of individual needs of the student ; (2) offer health education and teach emotional self-management competencies ; (3) increase victims´ knowledge of reactive (e.g. deleting, blocking and ignoring messages), preventive (e.g. increased awareness and security)  and effective strategies and resources that enable victims to cope with the experienced stress and negative emotions ; (4) aim at reducing traditional bullying as well , because victims are often involved in both forms of bullying [25, 26, 27]; and (5) include training in empathy, Internet etiquette, and healthy Internet behavior [28, 29]. Moreover, cyberbully victims often are unwilling to talk to a parent , teacher [2, 30] or other adults . They spend a lot of time online , prefer to get anonymous help , and report a need for information and help through the Internet . Therefore, the best method to deliver cyberbullying interventions is via the Internet. Furthermore, web-based interventions can be used whenever and wherever the individual prefers , can reach a lot of people in a relatively cheap way , and have the possibility to use tailoring , which appears to be a successful health promotion technique [38, 39].
To increase the effectiveness of such an intervention, a planned, systematic and theory-based approach is needed . Contrastingly, the interventions that currently exist are often based on practical beliefs or commonsense approaches, without a basis in theory or research results . A theoretically sound and evidence-based intervention should provide a description of what works, under what circumstances and for whom , with a thorough insight in the relevant determinants of behavioral change, the theoretical methods to change these determinants and how the theoretical methods are translated into practical intervention strategies [42, 43]. In order to meet the need for theoretically sound and evidence-based interventions, and based on theory and recommendations from the literature, we have developed an intervention containing three web-based and computer tailored pieces of advice for cyberbully victims: Online Pestkoppenstoppen (Stop online bullies/Stop bullies online). This paper focuses on the systematical and theory-based development of the intervention, using the Intervention Mapping (IM) protocol . Effects of the intervention are described elsewhere.
IM is a protocol consisting of six steps that can be used as an iterative process for theory and evidence-based development of health promotion interventions . The six steps include: (1) conducting a needs assessment of the problem, the study population and forming a logic model of the health problem based on the PRECEDE model ; (2) defining what program participants have to do in terms of performance objectives, and combining these performance objectives with relevant determinants into change objectives; (3) translating change objectives into practical strategies by selecting theory-based intervention methods; (4) developing, selecting, testing, and producing intervention components in which all strategies are integrated; (5) planning for adoption and implementation of the program; and (6) anticipating on the process and effect evaluation of the program. In this paper we describe how we used each step of the IM protocol to develop the intervention, focusing on the first four steps. We refer to traditional bullying as bullying, to cyberbullying and traditional bullying as (cyber)bullying, and to cyberbully victims and bully/victims as cyberbully victims. Further distinction are provided when necessary.
Step 1: Needs assessment
The target group for this intervention are adolescents (12-15 years old) from the Continued Secondary Vocational Education who are just starting to attend secondary school, because: (1) cyberbullying appears to occur more frequently in lower levels of secondary education [46, 47, 48]; (2) in this period of development the interaction with peers is highly valued and new social networks are formed ; and (3) adolescents have an open mind and are eager to learn new skills which enable them to learn how to cope more effectively with problems such as the negative effects of cyberbullying .
According to the Transactional Model , coping is the cognitive and behavioral effort employed to reduce, master or tolerate internal and external demands resulting from stressful events. In traditional bullying, victims either use more emotion focused [52, 53] and passive coping strategies (e.g. crying, expressing emotions) [54, 55, 56], or they use emotion focused and aggressive coping strategies (e.g. venting their anger, fighting back). Similarly, cyberbully victims report using aggressive and passive coping strategies  (e.g. bullying the bully, deleting messages or pretending to ignore the bullying ). Others react to cyberbullying by not retaliating and reacting submissively , acting helpless, avoiding the situation , doing nothing and displaying avoidant behavior . Some pretend to ignore the bullying, others actually ignore the it . Additionally, adolescents coping with cyberbullying are less likely to seek social support, compared to other strategies [30, 53, 58, 62]. Individuals that are bullied in multiple ways even tend to cope less effectively .
It appears that the use of ineffective coping strategies maintains (cyber)bullying [52, 54, 55, 56, 63, 64, 65, 66, 67], and that the negative effects of cyberbullying are influenced by the coping style victims use (e.g., ineffective coping appears to yield depression- and health complaints) [68, 69]. To increase knowledge about coping strategies used in cyberbullying, it is useful to look at coping strategies employed in cyberbullying and in daily life. These coping strategies appear to not differ significantly, and knowledge about traditional bullying can, to some extent, be applied to cyberbullying . Furthermore, coping strategies used in response to daily stressors are good predictors for coping strategies used in response to cyberbullying . To reduce victimization and its negative effects, it thus appears that adolescents need to improve their current coping strategies. They need to employ effective coping strategies that not only help them to mentally deal with (cyber)bullying, but also contribute to the prevention and discontinuation of (cyber)bullying. Therefore, the primary program goals were reducing the number of: (1) cyberbully victims; and (2) depressive and anxious victims as a consequence of cyberbullying. Secondary goals were a decrease in: (3) victims truanting from school; (4) victims with suicidal thoughts; and (5) an increase in determinants related to cyberbully victims’ behavior, such as self-esteem, self-efficacy, rational helpful beliefs, and effective coping.
Step 2: Matrices of change objectives
First, we translated the risk behavior ineffective coping into a program outcome: after the intervention, cyberbully victims will cope in an adequate and effective manner with (cyber)bullying experiences. This program outcome is a broad conceptualization of what a participant has to do in order to reach the program’s goals. Therefore, participants have to perform specific sub-behaviors, called performance objectives (POs). The POs, and accordingly the content of the intervention, were based on a literature search, focus group interviews with the target group and the linkage group (including the research team, a healthcare coordinator of a school community, a trainer who works with bullied children, and a project leader and director of a website addressing bullied children) formed for this project, a review of relevant (cyber)bullying websites, discussions in an online group about cyberbullying and cybersafety (YouthRiskOnline/Embracecivilty) and a Delphi study among 70 international experts in the field of cyberbullying and coping . In total, eleven performance objectives were formed for this intervention. For an overview of all POs see the first column of Additional file 1.
After thoughts, the 5G-schema focuses on emotions. Relational bullyvictims often have problems with the correct encoding and interpretation of emotions . It is important for an adolescent’s social and cognitive development to correctly perceive and attribute emotions . The understanding and experiencing of emotions of others seems to inhibit antisocial behavior , and adolescents who are able to perceive, handle, understand and express their emotions tend to experience better social relationships and are more socially accepted . Furthermore, despite bullyvictims probably being afraid, their most common reactions to victimization are interest and joy . Emotional displays of joy and interest can provide the bully with positive social reinforcement, leading to continued victimization. Therefore, (cyber)bullyvictims need to learn more about emotions and emotion regulation and need to receive health education and emotional self-management competencies  (PO 4).
The next aspect the 5G-schema focuses on is behavior. As explained before, whether an adolescent becomes a long-term victim of (cyber)bullying seems to depend on how (s)he copes with (cyber)bullying attempts. Sometimes, the coping strategies employed reduce stress, but the victim fails to confront the bully  leading to possible continuation of the bullying. Victims of cyberbullying therefore should become increasingly aware of the ineffectiveness of their current coping style (PO 5). They need knowledge of reactive and preventive strategies (PO 6) and should change their ineffective coping strategies into effective coping strategies (PO 7) . In this paper, all effective coping strategies are named Active Problem Solving (APS) strategies. APS strategies appear to resolve the bullying situation and allows the victim to act assertively . For example, cyberbully victims should try to get help from bystanders, because one quarter of cyberbullying occurs in the presence of witnesses . They should also block or ignore the cyberbully , or seek social support . Seeking social support from family is likely to help . Therefore adolescents need information regarding who to ask for help, how to seek help and the benefits of seeking help . Furthermore, having social skills is associated with reductions in victimization and fewer internalizing problems . Cross et al.  for example suggested that students should develop their social skills and learn effective ways of addressing relational difficulties online and offline, in order to prevent and reduce cyberbullying. Additionally, interventions should also focus on improving peer relationships in general , because social acceptance  and integration  moderate the impact and consequences of different forms of cyberbullying. Moreover, victims and adolescents not involved in bullying most frequently indicated assertiveness as an intervention strategy . Therefore, interventions should also teach (cyber)bullyvictims non-aggressive responses, and how to cope in an assertive and prosocial manner (PO 8 and 9).
A specific risk factor for cyberbullying is the little control exerted over personal information . A lot of cyberbully victims frequently use the Internet, most of the time in a risky manner [2, 78, 95]. They share passwords, openly display personal information (e.g. addresses, phone numbers) and communicate with strangers [96, 97]. Adolescents should therefore receive cyber safety education, learn to treat each other with respect (both online and offline), and become aware of the importance to regularly change their passwords and never sharing personal information  (PO 11).
The next step in IM is an analysis of the most relevant (i.e. important, changeable) determinants  of each PO, by asking what factors determine whether a victim would or is able to perform each PO. The methods used to obtain these determinants were the above mentioned literature review, focus group interviews, a Delphi study on the determinants of ineffective and improved coping with cyberbullying , and a study of protocols of (cyber) bullying interventions. Further, several behavior-oriented theories were applied, such as social cognitive theory, theory of planned behavior, transtheoretical model and goal-setting theory. This resulted in a large list of determinants (see the second column of Additional file 1). The aims of this intervention did not include environmental levels (e.g. parents, teachers or schools). Therefore only determinants on the individual level were identified.
A matrix was developed to combine performance objectives and their hypothesized determinants into change objectives (COs) . In total, we formed 50 COs. Examples of COs are ‘Individual describes characteristics of irrational and rational thoughts’ and ‘Individual becomes aware of current ineffective (aggressive/passive) coping style’ (see third column Additional file 1). All COs were used in the selection of theoretical methods and applications (step 3) that will change the determinants.
Step 3: Theory-based methods and practical strategies
To identify theoretical methods, parameters and practical strategies: (1) the literature was searched for behavior change techniques : (2) the IM book was used to find theoretical methods that influence specific determinant; (3) existing (cyber)bullying interventions were compared; (4) the PaS training  was used; and (5) focus group interviews were held, asking the members of the target group what they believed were effective strategies. An overview of theoretical methods and practical applications used in the Online Pestkoppenstoppen intervention is shown in the fifth column of Additional file 1. An elucidation of the most important methods follows next.
As mentioned before, tailoring appears to be a successful health promotion technique [38, 39]. Tailoring is the combination of strategies and information intended to reach one specific person by providing personalized feedback based on unique characteristics of a person. These characteristics are assessed individually and are related to the outcome of interest . The use of tailored information results in more improvement in behavior over time compared to the use of generic information [38, 102]. Tailored messages are better read, saved and remembered than non-tailored messages . There are two types of tailoring; dynamic tailoring (DT), assessing intervention variables prior to each feedback; and static tailoring (ST), assessing intervention variables at one baseline moment. Although research indicates that DT interventions outperform ST interventions , this intervention mostly tailors statical, because several variables used for tailoring are also used for the measurement of effectiveness, and thus need to be assessed prior to each advice. Tailoring is a general technique used throughout the complete intervention and is also combined with the other methods used. Prior to each advice several variables are assessed to enable tailoring. The intervention is for example tailored to personality (ST) as measured by the Big 5 Questionnaire –Short form . Personality may determine a participant’s behavior within a specific context, and this behavior (e.g. reactions, interpretations) can influence the reoccurrence of that event . Other variables on which the intervention provides tailored feedback are for example coping strategies (ST), irrational (ST) and rational (DT) thoughts, self-efficacy (DT), (cyber)bullying experiences (ST) and progress (ST).
Consciousness raising is used to increase a participant’s awareness about: (1) the thoughts (s)he has after a (cyber)bullying event; (2) his/her own negative 5G-schemas; (3) ineffective coping styles and negative outcomes; and (4) effective coping styles and positive outcomes. Based on answers about irrational thoughts and coping styles, the participant receives tailored feedback concerning his/her irrational thoughts, 5G-schemas, and current coping responses. This feedback may increase awareness, which in turn may help the participant to see why the current way of thinking and/or coping is not helpful or effective. It is important that the participant receives information about problem solving immediately . Therefore, the participant immediately receives information about how to recognize, dispute and replace thoughts after becoming aware of his/her 5G-schema, and how to change ineffective coping into effective coping after becoming aware of his/her ineffective coping.
Providing information is used to increase participant’s knowledge, skills, and attitudes, using arguments and persuasive communication . According to the elaboration likelihood model , good arguments are effective in reaching stable change in attitudes only when the message is processed through the central route. This happens when the message is personally relevant and not to discrepant from the participant’s beliefs . The participant receives general (e.g. what are the dynamics of a bullying situation, what to do and not to do online) and tailored (e.g. questions that aid recognizing irrational thoughts and changing aggressive/passive coping into active and problem solving coping) information. To check whether the participant remembers the correct information, questions are asked interspersed with feedback. Additionally, to prevent the participant from paying attention to information (s)he already knows, at some points in the intervention (s)he can choose which information (s)he wants to read. Furthermore, all important information is summarized on the personal page and sent to the participant’s e-mail inbox.
In modeling a person serves as an appropriate model for others that is reinforced for the desired behavior assuming that the behavior of the model will be imitated . Modeling is included in this intervention as a method to increase participant’s self-efficacy and behavioral capability to carry out the desired behavior change, because it is proven to be an effective teaching tool . For example, previous research indicated that video-counseling - in which a real human speaks and asks questions - can be a promising technique for health interventions . In line with the parameters of modeling, the video-models are coping models (i.e. who previously experienced the health problem, but who learned to deal with it effectively), the participant is able to identify with the models, and the models are being reinforced for positive behavior . The short videos were professionally recorded . In several video-clips, models demonstrate and explain how to perform a behavior (e.g. recognizing, disputing and replacing irrational thoughts, coping behaviors).
Implementation intentions and coping plans
Research indicates that planning is a powerful self-regulatory tool in the process of behavior change [111, 112]. When people plan, they mentally simulate the linking of concrete responses to future situations, replacing spontaneous reactions for pre-planned actions. There are two types of plans: action plans , or implementation intentions , and coping plans . In action planning, individuals link goal-directed behavior to certain environmental cues by specifying when, where and how to act  in if-then statements . As a result, environmental cues should activate - without conscious intent - the initiation of action . In coping planning, individuals anticipate on possible risk situations beforehand by already defining suitable and effective response behavior linked to these potential barriers . Implementation intentions, action and coping plans are found to be effective in translating various intentions into actions [116, 117, 118], and appear to be working synergistically in adolescents . In the intervention, the participant makes a combination of implementation intentions and coping plans. A closed-ended plan setting procedure is used to prevent inadequate goals; from a predefined list, the participant chooses the context if (either difficult situations or situations in daily life) and action then (applications of learned skills, from a predefined list).
Step 4: Producing program components and materials
Taking the end products of the previous steps into account, we decided to develop one intervention containing three web-based sessions containing tailored pieces of advice addressing mostly personal and psychological determinants. A program plan including the scope, sequence and program materials was developed , which is described below. During the process of producing program components and materials, intended program participants and the Linkage group were consulted to determine the preferences for program design by conducting focus group interviews and short voting sessions.
Scope and sequence
Because the literature indicates that an explicit approach (e.g. ‘did you experience cyberbullying?’) systematically underestimates the problem [47, 58], an implicit approach was used; the individual rates his or her experience with different cyberbullying behaviors. Individuals are included in the intervention if they had experiences with at least one of the cyberbullying behaviors at least ‘once a month’ during the last half year. Cyberbullies and adolescents who did not experience the 17 deviant cyberbehaviors are excluded from participation. Next, the individual (=participant) fills in his or her (nick)name, age, e-mail address, mobile phone number and a login code that is provided to him/her in an information letter. The e-mail address and mobile phone number will be used to remind participants of participation before the start of each advice, during the process of filling in the questionnaires, and during the process of completing each advice.
Below, we shortly describe the different materials developed for the intervention.
After selection/logging in, the participant is redirected to the personal page, from which (s)he: (1) can proceed or continue with the questionnaires and/or pieces of advice; (2) can find for each advice pictures, video-clips, comics, summaries, formed helpful thoughts, plans and sentences to use; and (3) has access to the extra tailored social skill lessons of advice 2.
Assessing tailoring and effect measures
Before receiving each web-based tailored advice, the participant answers several questionnaires (T0, T1 and T2) to assess the effect and tailoring variables. Gender of the participant was used to tailor gender of the guides and contexts (e.g. girls will see Noa, Jessica and a girl’s bedroom, and boys will see Levi, Mike and a boy’s bedroom).
Advice 1.‘Think strong, feel better.’
After the baseline measure (T0), the participant starts with the first advice. The participant learns to: (1) work with the 5G-schema in order to recognize (PO 1), dispute (PO 2) and replace (PO 3) irrational thoughts with rational thoughts; (2) carefully examine emotions in response to certain events (PO 4); and (3) form action plans to use the 5G-schema in daily life (PO 10).
Advice 2 ‘Stop the bully now!’
A month after the start, the participant begins with the second questionnaire (T1) and advice. The participant: (1) receives knowledge about the dynamics of online and offline bullying situations; (2) receives general APS coping strategies (PO 6); (3) learns and forms implementation intentions for responding towards online and offline bullying (PO 7 and PO 10); (4) becomes aware of ineffective coping style (PO 5); (5) is offered tailored examples of and guidance in APS coping (PO 6 and 7); (6) learns about responding assertively (PO 8 and 9); (7) forms implementation intentions and coping plans regarding assertive responses (PO 10); and (8) receives information about the use and benefits of humor and bystanders (PO 6). Observing victimization on its own can have negative consequences on psychological wellbeing . Therefore, the participant forms implementation intentions and coping plans regarding how to react and involve bystanders in (cyber)bullying situations (PO 10).
Social lessons based on YSR subscales with norm scores for gender
YSR social problems
Male > 7
Female > 7
Male > 6
Female > 7
Male > 12
Female > 13
Express feelings and opinions
Planning to become more socially active
Advice 3 ‘You are doing great, can you do better?’
Two months after the start, the participant starts with the third questionnaire (T2) and follow up advice. The participant evaluates all plans made in advice 1 and 2 on usability, and if necessary adjusts the plans (PO 10). All skills of advice 1 and 2 are summarized into three steps (i.e. 1. Think; 2. Act; 3.Closure). These three steps together form the skill conflict resolution, which is associated with reductions in victimization and fewer internalizing problems . Tailored on progress, the participant receives feedback (on irrational thoughts, coping responses and YSR scores) and if needed additional lessons (5G-schema, coping styles, humor, non-verbal communication and emotion regulation). Finally, the participant learns and plans how to use the Internet and mobile phone in a safe and secure manner (PO 11).
Pretest of the interventions
Parts of the focus group interviews conducted for the development of this intervention were pre- and pilot tests. Members of the target group were instructed to go to the website, complete the questionnaires and follow the pieces of advice. Next, six focus group interviews were held with a total of 41 participants from 4 schools. Based on the results from the pretests, pilot tests and focus group interviews, several adjustments were made: lay-out issues were resolved, bugs were repaired and both the number of questions and the amount of content were reduced.
Step 5: Adoption and Implementation
A plan for the adoption, implementation and continuation of the web-based tailored intervention was developed. To facilitate adoption and implementation, a linkage group was formed . The intervention was developed in such a way that the potential for adoption, implementation and continuation is enhanced. For example, intervention users can find, register and use the intervention online. Therefore, the implementation of the intervention does not need additional human actions. The only actions needed and costs associated with the implementation, adoption and continuation are recruiting participating schools or individuals, hosting the website, preparing the intervention for implementation after the research has been conducted, and keeping the information up-to-date. Hence, the intervention can easily reach a large number of adolescents and at very low costs.
Step 6: Planning for evaluation
The evaluation of the program - which is described below - was planned. The research on the effectiveness of the intervention Online Pestkoppenstoppen has been approved by the Medical Ethics Committee of the Atrium Hospital of Heerlen (NL39072.096.12; METC nr. 12-T-126), and is registered in the Dutch Trial Register (http://NTR3613).
Design and procedure
Participants will be recruited via contacting all secondary vocational education schools in the Netherlands. When schools give their consent to participation, randomization will follow. Schools will be stratified on three categories; a) size (small, medium or large); b) location of school (urban or rural); and c) type of education (theoretical or both theoretical and vocational). Schools will be matched and randomly assigned to one of the three intervention conditions. After randomization, all participants will attend a lesson at school in which the research will be explained. Participants receive a written information letter and informed consent, which has to be signed by both the participant and his or her parent(s). Next, participants have to visit the website and complete the selection questionnaire. When a participant meets the inclusion criteria, registration will follow and the research and intervention procedure will start.
A back-up plan was created in case the above-mentioned method of recruiting did not provide a sufficient number of participants. In case of insufficient participants, adolescents are recruited by placing a message on several popular websites often visited by adolescents and/or their parents. Additionally, recruitment messages are sent via Facebook, LinkedIn and Twitter. In this message individual adolescents are recruited to participate in the research by asking them to send us an e-mail with their name, age, name of school and level of education. Next, they receive instructions (written and on video), a written information letter and informed consent for them and their parents. In case parents read the message, they are asked to let their child participate in the research. Only after the adolescent provides us with the signed informed consents from them and their parents, will a login code be sent to the participant with which (s)he can access the intervention and/or research.
To evaluate the effectiveness of the intervention, several concepts will be measured via self-reported questionnaires. The primary outcomes of this study are frequency of (cyber)bullying, psychological wellbeing, problem behavior, school performance and truancy. These will be assessed at baseline (T0), 6 (T3) and 12 months (T4) follow-up. The secondary outcomes of this study (i.e. moderators or working mechanisms (mediators) of the intervention) are irrational thoughts, self-efficacy, coping with cyberbullying and self-esteem. These will be assessed at baseline (T0), after 1 (T1), 2 (T2), 6 (T3) and 12 months (T4) follow-up. At baseline, several moderators will be measured as well (i.e. demographic characteristics such as gender, age, educational level). In addition, a process evaluation at T3 will be conducted to examine potential moderators.
The power analysis is based on two important dependencies: decrease in psychosocial problems and the amount of cyberbullying victims after the intervention. The outcomes are based on the PaS training . A power calculation (effect size = .3; power = .80; intracluster correlation coefficient = .1) showed that at baseline about 446 participants were needed (based on a dropout rate of 20%). Based on previous studies we suspect that 25% of the adolescents between 12 and 15 years can be classified as cyberbullying victims, and that 50% of the adolescents are willing to participate. This results in 3570 adolescents, divided over 30 schools, with each school having 4 or 5 classes.
The aim of this paper was to provide a detailed description of the systematic development of the three web-based tailored pieces of advice for cyberbully victims that aimed at increasing psychological wellbeing, and decreasing victimization, problem behavior, school problems and truancy. In this intervention, cyberbully victims learn how to recognize, dispute and replace irrational thoughts with rational thoughts. Furthermore, their ineffective coping strategies are replaced with effective coping strategies, and they learn how to use the Internet and mobile phone in a safe and secure manner. The intervention is delivered online, via digital guides in digital environments. Digital guides provide information, ask questions, and show videos, pictures, comics and animations.
This intervention has several strengths. First, in developing, implementing and evaluating a systematic approach , the Intervention Mapping approach is used. In the process of developing the intervention, empirical evidence, theories, guidelines and recommendations from the cyberbullying and bullying intervention and prevention literature were used to build a solid framework. Second, the intervention aims to change multiple determinants that were found in a study done by Jacobs et al., . Third, guidelines and recommendations from other, not related to (cyber)bullying, intervention studies and theories were used, combined with results from focus groups interviews with the target groups. Fourth, this intervention is tailored to the specific needs of the target group: cyberbully victims are often unwilling to tell a teacher, parent or other adult that they need help. Therefore, they may feel more comfortable to  find help anonymously on the Internet . Furthermore, we tried to incorporate all preferences of the target group as indicated in the pre-tests and pilot tests. Fifth, web-based computer tailored interventions can be used any time , can reach a lot of people in a relatively cheap way , and allow for tailoring , which has the potential for a successful health promotion technique [38, 39]. Sixth, the planned RCT allows several important conclusions to be drawn. Finally, a process evaluation allows us to draw conclusions about several working mechanisms.
This intervention has a number of limitations. First, it was difficult to reduce the number of questions in the questionnaires we used for evaluation purposes. In the focus group interviews, members of the target group clearly stated that too many questions will lead to drop-out, because answering many questions is experienced as boring and fatiguing. Furthermore, too many questions may cause no-saying or yes-saying and response fatigue (for a review, see ), leading to less reliable results. Second, the intervention may contain too much information. Based on the literature, it was decided to include the 5G-schema, coping strategies, social skills, planning and safety instructions. In focus group interviews, members of the target-group mentioned the large amount of information making it difficult to remember everything. Third, we cannot exactly check the degree of usage on the experimental and general information. This may hamper the interpretation of the results. Therefore, we have added process evaluation questions, in which we ask the participants if they have read the information.
In conclusion, the Online Pestkoppenstoppen program seems to be a promising start in solving current cyberbullying problems. The results of the evaluation studies, which will be reported in other papers, may contribute to the knowledge about how to stop online victimization and how to use tailoring and web-based interventions in this purpose. Additionally, this intervention may also affect offline victimization, because despite the differences, cyberbullying and traditional bullying also have some overlapping characteristics [24, 25, 27]. The results can be used as input for other web-based and computer-tailored interventions aimed at cyberbully victims. If the studies point out that the intervention is effective in its purpose, we have an intervention ready for implementation on a large scale.
This work was supported by the Dutch Ministry of Education, Culture and Science [grant number ODB10001]. We would like to thank Peter-Paul Lenssen from the Bernardinuscollege for providing the filming locations and actors, and Jetze Jansma from Kumulus Jeugdtheater for providing actors. Additionally, we would like to thank Pepijn van Halderen, Joelle Canisius, Pim Hovens, Yorn Heijnen, Melissa Vermeulen, Julie Cosemans, Vera Nijssen, Luca Meisters, Benno Geurts, Iris Voogt and Sophie Berg for doing such a great job acting in the movies. We also would like to thank Sophie Mulder, Jaron Bos, Saskia Wouters, Rick Jacobs and Janneke Duker for providing their voices, and Marcel Maassen for recording and editing the voice recordings. A big thank you goes to Sandra Daems and Annette Bouwels-Hermens for recording and editing several extra movie clips that were used in the recruitment of schools and introducing the participants with the intervention. We would also like to thank Nikki Gerits and Steffi Bekkers for conducting two focus group interviews. Finally, we would like to thank Linda Goossens (research assistant for this project), for all of her hard work and excellent planning and organizing skills.
- 1.Gross EF: Adolescent Internet use: What we expect, what teens report. J Appl Dev Psychol. 2004, 25 (6): 633-649. 10.1016/j.appdev.2004.09.005.Google Scholar
- 3.Tokunaga RS: Following you home from school: A critical review and synthesis of research on cyberbullying victimization. Comput Hum Behav. 2010, 26 (3): 277-287. 10.1016/j.chb.2009.11.014.Google Scholar
- 4.Perren S, Dooley J, Shaw T, Cross D: Bullying in school and cyberspace: Associations with depressive symptoms in Swiss and Australian adolescents. Child Adolesc Psychiat Ment Health. 2010, 4 (1): 1-10. 10.1186/1753-2000-4-1.Google Scholar
- 6.Campbell M, Spears B, Slee P, Butler D, Kift S: Victims’ perceptions of traditional and cyberbullying, and the psychosocial correlates of their victimisation. Emotional Behav Difficulties. 2012, 17 (3–4): 389-401.Google Scholar
- 9.Beran T, Li Q: The relationship between cyberbullying and school bullying. J Stud Wellbeing. 2007, 1 (2): 15-33.Google Scholar
- 10.Katzer C, Fetchenhauer D, Belschak F: Cyberbullying: who are the victims? a comparison of victimization in internet chatrooms and victimization in school. J Media Psychol. 2009, 21 (1): 25-36. 10.1027/1864-1220.127.116.11.Google Scholar
- 12.Ybarra ML: Linkages between depressive symptomatology and Internet harassment among young regular internet users. Cyber Psychol Behav. 2004, 7 (2): 247-257. 10.1089/109493104323024500.Google Scholar
- 14.Ybarra ML, Mitchell KJ, Wolak J, Finkelhor D: Examining characteristics and associated distress related to internet harassment: findings from the second youth internet safety survey. Pediatrics. 2007, 118 (4): 1169-1177.Google Scholar
- 15.Beran T, Li Q: Cyber-harassment: a study of a new method for an old behavior. J Educ Comput Res. 2005, 32 (3): 265-277. 10.2190/8YQM-B04H-PG4D-BLLH.Google Scholar
- 17.Schoffstall CL, Cohen R: Cyber aggression: The relation between online offenders and offline social competence. Soc Dev. 2011, 20 (3): 587-604. 10.1111/j.1467-9507.2011.00609.x.Google Scholar
- 18.Låftman SB, Modin B, Östberg V: Cyberbullying and subjective health: A large-scale study of students in Stockholm, Sweden. Child Youth Serv Rev. 2013, 35 (1): 112-119. 10.1016/j.childyouth.2012.10.020.Google Scholar
- 19.Slonje R, Smith PK, Frisén A: The nature of cyberbullying, and strategies for prevention. Comput Hum Behav. 2013, 29 (1): 26-32. 10.1016/j.chb.2012.05.024.Google Scholar
- 20.Paul S, Smith PK, Blumberg HH: Comparing student perceptions of coping strategies and school interventions in managing bullying and cyberbullying incidents. Pastoral Care in Educ. 2012, 30 (2): 127-146. 10.1080/02643944.2012.679957.Google Scholar
- 21.Snakenborg J, Van Acker R, Gable RA: Cyberbullying: Prevention and Intervention to Protect Our Children and Youth. Prev Sch Fail. 2011, 55 (2): 88-95. 10.1080/1045988X.2011.539454.Google Scholar
- 22.Patchin JW, Hinduja S: Traditional and Nontraditional Bullying Among Youth: A Test of General Strain Theory. Youth Soc. 2011, 43 (2): 727-751. 10.1177/0044118X10366951.Google Scholar
- 23.Tenenbaum LS, Varjas K, Meyers J, Parris L: Coping strategies and perceived effectiveness in fourth through eighth grade victims of bullying. Sch Psychol Int. 2011, 32 (3): 263-287. 10.1177/0143034311402309.Google Scholar
- 24.Olweus D: Invited expert discussion paper Cyberbullying: An overrated phenomenon?. Eur J Dev Psychol. 2012, 9 (5): 1-19.Google Scholar
- 25.Hinduja S, Patchin JW: Cyberbullying: An Exploratory Analysis of Factors Related to Offending and Victimization. Deviant Behav. 2008, 29 (2): 129-156. 10.1080/01639620701457816.Google Scholar
- 26.Hemphill SA, Kotevski A, Tollit M, Smith R, Herrenkohl TI, Toumbourou JW, Catalano RF: Longitudinal Predictors of Cyber and Traditional Bullying Perpetration in Australian Secondary School Students. J Adolesc Health. 2012, 51 (1): 59-65. 10.1016/j.jadohealth.2011.11.019.PubMedPubMedCentralGoogle Scholar
- 27.Casas JA, Del Rey R, Ortega-Ruiz R: Bullying and cyberbullying: Convergent and divergent predictor variables. Comput Hum Behav. 2013, 29 (3): 580-587. 10.1016/j.chb.2012.11.015.Google Scholar
- 29.Barlińska J, Szuster A, Winiewski M: Cyberbullying among adolescent bystanders: Role of communication medium, form of violence, and empathy. J Commun Applied Social Psychol. 2013, 23: 37-51. 10.1002/casp.2137.Google Scholar
- 31.Ahlfors R: Many Sources, One Theme: Analysis of Cyberbullying Prevention and Intervention Websites. J Soc Sci. 2010, 6 (4): 515-522.Google Scholar
- 32.Lenhart A, Purcell K, Smith A, Zickuhr K: Social Media & Mobile Internet Use among Teens and Young Adults. 2010, Washington, DC: Pew Research Center, Pew Internet & American Life ProjectGoogle Scholar
- 34.Havas J, de Nooijer J, Crutzen R, Feron F: Adolescents' views about an internet platform for adolescents with mental health problems. Health Educ. 2011, 111 (3): 164-176. 10.1108/09654281111123466.Google Scholar
- 35.Oenema A, Brug J, Dijkstra A, Weerdt I, Vries H: Efficacy and Use of an Internet-delivered Computer-tailored Lifestyle Intervention, Targeting Saturated Fat Intake, Physical Activity and Smoking Cessation: A Randomized Controlled Trial. Ann Behav Med. 2008, 35 (2): 125-135. 10.1007/s12160-008-9023-1.PubMedGoogle Scholar
- 41.Smith PK, Salmivalli C, Cowie H: Effectiveness of school-based programs to reduce bullying: a commentary. J Exp Criminol. 2012, 8 (4): 433-441. 10.1007/s11292-012-9142-3.Google Scholar
- 42.Michie S, Johnston M, Francis J, Hardeman W, Eccles M: From Theory to Intervention: Mapping Theoretically Derived Behavioural Determinants to Behaviour Change Techniques. Appl Psychol. 2008, 57 (4): 660-680. 10.1111/j.1464-0597.2008.00341.x.Google Scholar
- 43.Webb TL, Joseph J, Yardley L, Michie S: Using the internet to promote health behavior change: a systematic review and meta-analysis of the impact of theoretical basis, use of behavior change techniques, and mode of delivery on efficacy. J Med Internet Res. 2010, 12 (1): e4-10.2196/jmir.1376.PubMedPubMedCentralGoogle Scholar
- 44.Bartholomew LK, Parcel GS, Kok G, Gottlieb NH, Fernández ME: Planning Health Promotion Programs: An Intervention Mapping Approach. 2011, San Francisco: Jossey-BassGoogle Scholar
- 45.Green LW, Kreuter MW: Health Program Planning: An Educational And Ecological Approach. 2005, New York: McGraw-HillGoogle Scholar
- 46.van der Vegt AL, den Blanken M, Jepma I: Nationale scholierenmonitor: meting voorjaar 2007. Onderwijsvernieuwingenrosernl. 2007, http://www.hetccv.nl/dossiers/geweld-op-school/landelijk---nationale-scholierenmonitor-2007,Google Scholar
- 47.Walrave M, Heirman W: Cyberbullying: Predicting Victimisation and Perpetration. Child Soc. 2011, 25 (1): 59-72. 10.1111/j.1099-0860.2009.00260.x.Google Scholar
- 48.Wade A, Beran T: Cyberbullying: The new era of bullying. Can J Sch Psychol. 2011, 26 (1): 44-61. 10.1177/0829573510396318.Google Scholar
- 49.Gavin LA, Furman W: Age differences in adolescents' perceptions of their peer groups. Dev Psychol. 1989, 25 (5): 827-Google Scholar
- 50.Faber M, Verkerk G, van Aken M, Lissenburg L, Geerlings M: Plezier op school: sterker naar de brugklas. Kind en Adolescent Praktijk. 2006, 5 (1): 31-39. 10.1007/BF03059577.Google Scholar
- 51.Lazarus RS, Folkman S: Stress, appraisal, and coping. 1984, New York: SpringerGoogle Scholar
- 52.Craig W, Pepler D, Blais J: Responding to bullying: what works. Sch Psychol Int. 2007, 28 (4): 465-477. 10.1177/0143034307084136.Google Scholar
- 53.Kowalski RM, Limber SP, Agatston PW: Cyber Bullying: Bullying in the digital age. 2008, Malden: Blackwell PublishingGoogle Scholar
- 54.Mahady Wilton MM, Craig WM, Pepler DJ: Emotional regulation and display in classroom victims of bullying: characteristic expressions of affect, coping styles and relevant contextual factors. Soc Dev. 2000, 9 (2): 226-245.Google Scholar
- 57.Dooley JJ, Shaw T, Cross D: The association between the mental health and behavioural problems of students and their reactions to cyber-victimization. Eur J Dev Psychol. 2012, 9 (2): 275-289. 10.1080/17405629.2011.648425.Google Scholar
- 58.Dehue F, Bolman C, Völlink T: Cyberbullying: youngsters’ experiences and parental perception. Cyber Psychol Behav. 2008, 11 (2): 217-223. 10.1089/cpb.2007.0008.Google Scholar
- 59.Acirak OT: Psychiatric symptomatology as predictor of cyberbullying among university students. Eurasian J Educ Res. 2009, 34: 167-184.Google Scholar
- 60.Riebel JRSJ, Fischer UC: Cyberbullying in Germany – an exploration of prevalence, overlapping with real life bullying and coping strategies. Psychol Sci Q. 2009, 51 (3): 298-314.Google Scholar
- 61.Perren S, Corcoran L, Cowie H, Dehue F, Garcia D, Mc Guckin C, Sevcikova A, Tsatsou P, Völlink T: Tackling cyberbullying: Review of empirical evidence regarding successful responses by students, parents, and schools. Int J Conflict Violence. 2012, 6 (2): 283-292.Google Scholar
- 62.Parris L, Varjas K, Meyers J, Cutts H: High School Students’ Perceptions of Coping With Cyberbullying. Youth Soc. 2012, 44 (2): 284-306. 10.1177/0044118X11398881.Google Scholar
- 63.Skrzypiec G, Slee P, Murray-Harvey R, Pereira B: School bullying by one or more ways: Does it matter and how students cope?. Sch Psychol Int. 2011, 32 (3): 288-311. 10.1177/0143034311402308.Google Scholar
- 64.Andreou E: Bully/Victim Problems and their Association with Coping Behaviour in Conflictual Peer Interactions Among School-age Children. Educ Psychol. 2001, 21 (1): 59-66. 10.1080/01443410125042.Google Scholar
- 66.Perry DG, Hodges EVE, Egan SK, Juvonen J, Graham S: Determinants of Chronic Victimization by Peers: A Review and New Model of Family Influence. Peer Harassment in School: The Plight of the Vulnerable and Victimized. 2001, New York, NY, US: Guilford Press, 73-104.Google Scholar
- 67.Kanetsuna T, Smith PK, Morita Y: Coping with bullying at school: children's recommended strategies and attitudes to school-based interventions in England and Japan. Aggress Behav. 2006, 32 (6): 570-580. 10.1002/ab.20156.Google Scholar
- 68.Völlink T, Bolman CAW, Dehue F, Jacobs NCL: Coping with Cyberbullying: Differences Between Victims, Bully-victims and Children not Involved in Bullying. J Commun Applied Soc Psychol. 2013, 23 (1): 7-24. 10.1002/casp.2142.Google Scholar
- 69.Neary A, Joseph S: Peer victimization and its relationship to self-concept and depression among school girls. Personal Individ Differ. 1994, 16 (1): 183-186. 10.1016/0191-8869(94)90122-8.Google Scholar
- 71.Cook CR, Williams KR, Guerra NG, Kim TE, Sadek S: Predictors of bullying and victimization in childhood and adolescence: a meta-analytic investigation. Sch Psychol Q. 2010, 25 (2): 65-83.Google Scholar
- 72.Bauman S: Cyberbullying in a rural intermediate school: An exploratory study. J Early Adolesc. 2010, 30 (6): 803-833. 10.1177/0272431609350927.Google Scholar
- 73.Ellis A: The essence of RET. J Ration Emot Cogn Behav Ther. 1984, 2 (1): 19-25. 10.1007/BF02283005.Google Scholar
- 74.Ellis A: Changing Rational-emotive therapy (RET) to Rational, emotive behavior therapy (REBT). J Ration Emot Cogn Behav Ther. 1995, 13 (2): 85-89. 10.1007/BF02354453.Google Scholar
- 75.Ringrose HJ, Nijenhuis EH: Bang zijn voor andere kinderen. 1986, Groningen: Wolters-NoordhoffGoogle Scholar
- 76.Ellis A: Overcoming Destructive Beliefs, Feelings and Behaviors: New Directions for Rational Emotive Therapy. 2001, New York: Prometheus BooksGoogle Scholar
- 78.Erdur-Baker Ö: Cyberbullying and its correlation to traditional bullying, gender and frequent and risky usage of internet-mediated communication tools. New Media Soc. 2010, 12 (1): 109-125. 10.1177/1461444809341260.Google Scholar
- 79.Hinduja S, Patchin JW: Offline consequences of online victimization. J Sch Violence. 2007, 6 (3): 89-112. 10.1300/J202v06n03_06.Google Scholar
- 81.Gonzalez JE, Nelson JR, Gutkin TB, Saunders A, Galloway A, Shwery CS: Rational Emotive Therapy With Children and Adolescents: A Meta-Analysis. J Emot Behav Disord. 2004, 12 (4): 222-235. 10.1177/10634266040120040301.Google Scholar
- 82.Banks T, Zionts P: REBT Used with Children and Adolescents who have Emotional and Behavioral Disorders in Educational Settings: A Review of the Literature. J Ration Emot Cogn Behav Ther. 2009, 27 (1): 51-65. 10.1007/s10942-008-0081-x.Google Scholar
- 84.Nowicki S, Duke MP: Individual differences in the nonverbal communication of affect: The Diagnostic Analysis of Nonverbal Accuracy Scale. J Nonverbal Behav. 1994, 18 (1): 9-35. 10.1007/BF02169077.Google Scholar
- 85.Kokkinos C, Kipritsi E: The relationship between bullying, victimization, trait emotional intelligence, self-efficacy and empathy among preadolescents. Soc Psychol Educ. 2011, 15 (1): 41-58.Google Scholar
- 86.Austin EJ, Saklofske DH, Egan V: Personality, well-being and health correlates of trait emotional intelligence. Personal Individ Differ. 2005, 38 (3): 547-558. 10.1016/j.paid.2004.05.009.Google Scholar
- 88.Price M, Dalgleish J: Cyberbullying: experiences, impacts and coping strategies as described by Australian young people. Youth Stud Australia. 2010, 29 (2): 51-59.Google Scholar
- 89.Fanti KA, Demetriou AG, Hawa VV: A longitudinal study of cyberbullying: Examining riskand protective factors. Eur J Dev Psychol. 2012, 9 (2): 168-181. 10.1080/17405629.2011.643169.Google Scholar
- 90.Kochenderfer-Ladd B: Peer Victimization: The Role of Emotions in Adaptive and Maladaptive Coping. Soc Dev. 2004, 13 (3): 329-349. 10.1111/j.1467-9507.2004.00271.x.Google Scholar
- 91.Cross D, Monks H, Campbell MA, Spears B, Slee P: School-based strategies to address cyber bullying. Centre for Strategic Education Occasional Papers. 2011, 119: 1-12.Google Scholar
- 92.Boulton MJ, Smith PK, Cowie H: Short-Term Longitudinal Relationships Between Children’s Peer Victimization/Bullying Experiences and Self-Perceptions Evidence for Reciprocity. Sch Psychol Int. 2010, 31 (3): 296-311. 10.1177/0143034310362329.Google Scholar
- 94.Camodeca M, Goossens FA: Children's opinions on effective strategies to cope with bullying: The importance of bullying role and perspective. Educ Res. 2005, 47 (1): 93-105. 10.1080/0013188042000337587.Google Scholar
- 95.Vandebosch H, van Cleemput K: Cyberbullying among youngsters: profiles for bullies and victims. New Media Soc. 2009, 11 (8): 1349-1371. 10.1177/1461444809341263.Google Scholar
- 96.Hinduja S, Patchin JW: Bullying beyond the Schoolyard: Preventing and Responding to Cyberbullying. 2009, California: Corwin pressGoogle Scholar
- 97.Valcke M, De Wever B, Van Keer H, Schellens T: Long-term study of safe Internet use of young children. Comput Educ. 2011, 57 (1): 1292-1305. 10.1016/j.compedu.2011.01.010.Google Scholar
- 99.Bandura A: Social Foundations of Thought and Action: a Social Cognitive Theory. 1986, Englewood Cliffs, NJ: Prentice HallGoogle Scholar
- 100.Bandura A: Perceived Self-Efficacy in Cognitive Development and Functioning. Educ Psychol. 1993, 28 (2): 117-148. 10.1207/s15326985ep2802_3.Google Scholar
- 101.Kreuter M, Farrell D, Olevitch L, Brennan L: Tailoring Health Messages: Customizing Communication with Computer Technology. 2000, Mahwah, NJ: ErlbaumGoogle Scholar
- 103.Brug J, Oenema A, Campbell M: Past, present, and future of computer-tailored nutrition education. Am J Clin Nutri. 2003, 77 (4): 1028S-1034S.Google Scholar
- 105.Caspi A, Bem DJ: Personality Continuity and Change across the Life Course. Handbook of Personality: Theory and Research. Edited by: Pervin LA. 1990, New York: Guilford Press, 549-575.Google Scholar
- 106.Petty RE, Cacioppo JT: The Elaboration Likelihood Model of Persuasion. Communication and Persuasion. 1986, New York: Springer, 1-24.Google Scholar
- 107.Bandura A: Self-efficacy: The exercise of control. 1997, New York: W.H. FreemanGoogle Scholar
- 108.Woolfolk A: Educational Psychology. 2010, Boston: Allyn & BaconGoogle Scholar
- 111.Sniehotta FF, Scholz U, Schwarzer R: Bridging the intention–behaviour gap: Planning, self-efficacy, and action control in the adoption and maintenance of physical exercise. Psychol Health. 2005, 20 (2): 143-160. 10.1080/08870440512331317670.Google Scholar
- 112.Sniehotta FF, Schwarzer R, Scholz U, Schüz B: Action planning and coping planning for long-term lifestyle change: theory and assessment. Eur J Soc Psychol. 2005, 35 (4): 565-576. 10.1002/ejsp.258.Google Scholar
- 114.Gollwitzer PM, Sheeran P: Implementation intentions: Bibliothek der Universität Konstanz. 2008Google Scholar
- 115.Sheeran P, Webb TL, Gollwitzer PM: The interplay between goal intentions and implementation intentions. Personal Soc Psychol Bull. 2005, 31 (1): 87-98. 10.1177/0146167204271308.Google Scholar
- 118.Wiedemann AU, Lippke S, Reuter T, Ziegelmann JP, Schwarzer R: How planning facilitates behaviour change: Additive and interactive effects of a randomized controlled trial. Eur J Soc Psychol. 2011, 41 (1): 42-51. 10.1002/ejsp.724.Google Scholar
- 120.Rivers I, Poteat PV, Noret N, Ashurst N: Observing bullying at school: The mental health implications of witness status. Sch Psychol Q. 2009, 24 (4): 211-Google Scholar
- 121.Achenbach TM: Integrative Guide for the 1991 CBCL/4-18, YSR, and TRF Profiles: Department of Psychiatry. 1991, Burlington: Department of Psychiatry, University of Vermont BurlingtonGoogle Scholar
- 122.Geerlings M, Lissenburg L: Evaluatieonderzoek naar de effectiviteit van de cursus ‘Plezier op school’. 2005, Utrecht: Universiteit UtrechtGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2458/14/396/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/4.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.