Adolescent Research Review

, Volume 3, Issue 2, pp 219–233 | Cite as

Technology-Oriented Suicide Prevention Interventions for Adolescents and Adolescent Gatekeepers: A Qualitative Review

Qualitative Review
  • 388 Downloads

Abstract

Youth suicide is increasing in the United States. To moderate youth suicide, it is important to implement effective prevention programs and target modifiable protective and risk factors through intervention. This review examined programs that are consistent with these goals, specifically, technology-oriented suicide prevention programs included in the National Registry of Evidence-Based Programs and Practices (NREPP) and Best Practices Registry (BPR). Technology-oriented programs are highly accessible among adolescents, as well as adolescent gatekeepers. Gatekeepers were defined as adults in a community who frequently interact with youth in the community, and who may be trained to identify and refer at-risk youth (e.g., teachers, coaches, counselors, parents). To understand the impact of these interventions, program efficacy (i.e., outcomes), program reach (i.e., population-level impact, level of technology integration), location of supporting program evidence (i.e., medical literature, Google Scholar, national registries, program organizational website), and quality of supporting research evidence (i.e., study design, quality of research ratings) were explored. PubMed, PsycINFO, PsycARTICLES, PASCAL, CINAHL, Scopus, Cochrane Library (n = 3,942); Google Scholar (n = 411); the NREPP (n = 127); the BPR (n = 114); and program organizational websites (n = 26) were searched. Published and unpublished studies (n = 35) were integrated. Review of technology-oriented interventions indicated that many have been found to improve secondary outcomes, suicide outcomes, and gatekeeper preparedness. Interventions also demonstrated potential for reach, as technology generally enhanced the spread of prevention content among adolescents and/or adolescent gatekeepers. However, the accessibility of evidence was often reduced through dispersion, as evidence was retained within multiple forums (i.e., medical literature, Google Scholar, national registries, program organizational websites). Finally, the quality of supporting research evidence was generally strong, although there was variability with regard to rigor in study design and inclusion of large samples. Additional research is important toward strengthening the evidence base, and additional evidence that is readily accessible may increase both reach and sustained use, to enhance overall impact.

Keywords

Technology Adolescent Gatekeeper Suicide prevention Intervention 

Introduction

Suicide in the United States is increasing (Centers for Disease Control and Prevention [CDC], 2016c). Suicide is the second leading cause of death among youth and emerging adults in particular (CDC 2016a). Age-adjusted suicide in the US increased 24% between 1999 and 2014 (CDC 2016c). The largest increases occurred in suicide among girls ages 10–14, with an increase of 200% between 1999 and 2014 (CDC 2016c). Annually, over 150,000 youth ages 10–24 years receive emergency treatment following self-inflicted injuries (CDC 2015). However, suicide data alone underestimate the true extent of this public health problem, as the number of youth who experience suicidal ideation, suicidal planning, and suicidal behavior far exceeds the number of youth who die by suicide (CDC 2016a). As a result, suicidal ideation, suicidal planning, and non-fatal suicidal behaviors encompass the “unseen” burden of adolescent suicide.

Because youth suicide imposes profound burdens, maintaining a public health approach provides a wide prevention-oriented lens that increases awareness of the many factors contributing to circumstances that promote suicidal ideation and suicidal behaviors (CDC 2016a; National Action Alliance for Suicide Prevention [NAASP] 2014). In particular, one important factor that contributes to suicide risk includes mental disorders. Half of lifetime cases of mental disorders begin by age 14, and 75% begin by age 24 (National Alliance on Mental Illness [NAMI] 2016). However, the average delay between symptom onset and intervention is 8–10 years (NAMI 2016). Prevention and intervention during adolescence is important because youth carry mental disorders into adulthood, which elevates suicide vulnerability. Screening and intervention during adolescence may prevent these disorders in adulthood, or may delay their onset (CDC 2016b).

Therefore, it is important to review the evidence base supporting highly accessible interventions and training resources that are designed to prevent suicide and target modifiable protective and risk factors (CDC 2016a, b; NAASP 2014). A public health approach is important because it permits integrated collaborative efforts among diverse partners (CDC 2016a, b; NAASP 2014). Because integrated collaborative efforts may increasingly moderate youth suicide, it is important to involve diverse community partners, such as adults who have proximity to and frequent interaction with youth (e.g., parents, school teachers, school counselors, coaches, club leaders). Directly involving youth and their peers is also important. Further, because youth suicide is a multifaceted public health problem, innovative solutions to suicide prevention, such as inclusion of technology-oriented resources, should also be explored.

Two national registries were created to highlight suicide prevention programs for adolescents and adolescent gatekeepers (i.e., individuals in a community who have face-to-face contact with large numbers of community members as part of their usual routine, and who may be trained to identify persons at risk of suicide and refer them to treatment or supporting services; NAASP 2014). Specifically, the Substance Abuse and Mental Health Services Administration (SAMHSA 2016) National Registry of Evidence-Based Programs and Practices (NREPP) includes 22 suicide prevention interventions; nine are designed for adolescents, or adults who serve as gatekeepers to adolescents, and leverage technology for optimal accessibility, scalability, and sustainability. Additionally, the Suicide Prevention Resource Center (SPRC 2016) Best Practices Registry (BPR) contains 114 suicide prevention programs, practices, and informational materials; 18 are designed for adolescents, or adults who serve as gatekeepers to adolescents, and leverage technology.

Adolescents and adolescent gatekeepers (e.g., parents, school teachers, school counselors, coaches, club leaders) have near universal access to technology-oriented resources, and technology has resultantly been leveraged in several adolescent and adolescent gatekeeper interventions. This qualitative review specifically explores these technology-oriented suicide prevention programs for adolescents and adolescent gatekeepers contained within the National Registry of Evidence-Based Programs and Practices and Best Practices Registry. The first aim was to identify the efficacy and reach of technology-oriented suicide prevention interventions. The second aim was to identify the accessibility of supporting program evidence and to assess the quality of supporting research evidence.

Current Review

As previously mentioned, there are two primary national suicide prevention registries, the Substance Abuse and Mental Health Service Administration’s National Registry of Evidence-Based Programs and Practices and the Suicide Prevention Resource Center’s Best Practices Registry. These are two distinct national databases. The National Registry of Evidence-Based Programs and Practices contains Section I suicide prevention programs, which are programs that have been evaluated for evidence of effectiveness, and that have produced at least one positive outcome related to suicide prevention during research (SAMHSA 2016). The Best Practices Registry includes Section III suicide prevention programs and practices, which are not examined for evidence of effectiveness; rather, these are programs and practices whose content have been reviewed for adherence to standards of safety, accuracy, likelihood of meeting objectives, programmatic guidelines, and messaging guidelines (SPRC 2016).

The Substance Abuse and Mental Health Services Administration (2016) independently and rigorously evaluate the content of “Legacy Programs” (i.e., reviewed prior to September 2015) and “Newly Reviewed” programs (i.e., reviewed after September 2015 using updated criteria), as well as select studies evaluating these programs. The National Registry of Evidence-Based Programs and Practices ratings reflect the strength of the conceptual framework, intervention effects on individual outcomes, ability of the intervention to achieve stated goals, degree to which implementation occurred as designed, quality of research examining the intervention, and resources available for dissemination and implementation. “Legacy Programs” contain quality of research ratings, which are based on a scale ranging from 0.0 to 4.0; in contrast, “Newly Reviewed” programs provide evidence ratings by outcome, identifying programs as having “effective,” “promising,” “ineffective,” or “inconclusive” outcomes (SAMHSA 2016).

The Suicide Prevention Resource Center (2016) maintains the Best Practices Registry, providing consolidated and integrated search capabilities of suicide prevention programs, practices, and materials. To be considered for inclusion in the Best Practices Registry, programs, practices, and materials must be reviewed for adherence to standards of safety, accuracy, likelihood of meeting objectives, programmatic guidelines, and messaging guidelines. Programs, practices, and materials must comply with 15 criteria related to accuracy, likelihood of meeting objectives, and consistency with programmatic and messaging guidelines to be included in the registry (SPRC 2016).

For this review the national registries are specifically explored because they represent the national platforms highlighting suicide prevention programs, practices, and materials for diverse stakeholders (e.g., researchers, policy makers, administrators, health professionals, schools, youth organizations, community groups). The efficacy of these interventions (i.e., program outcomes), the reach of these interventions (i.e., population-level impact, level of technology integration), the location of supporting intervention evidence (i.e., medical literature, Google Scholar, national registries, program organizational website), and the quality of supporting research evidence (i.e., study design, quality of research ratings) are explored in this review. The primary aim is to explore program efficacy and program reach. The second aim is to explore the quality of research examining the interventions. The location of supporting program evidence is also explored to describe the accessibility of interventions, and their ability to reach diverse stakeholders and the target population (i.e., adolescents and adolescent gatekeepers). The potential public health impact of interventions largely hinges on both efficacy and accessibility.

This review advances knowledge because the Best Practices Registry does not evaluate evidence of program effectiveness, and the National Registry of Evidence-Based Programs and Practices requires only one positive outcome even where the body of evidence is inconclusive. The Best Practices Registry also does not evaluate the quality of research supporting each intervention, which is explored in this review. The location of supporting program evidence is also explored, as centrally located and highly visible evidence partially ensures efficacious programs that are supported by quality research are accessible to intended stakeholders. Finally, technology-oriented programs are specifically explored because technology is highly accessible among and well received by adolescents and adolescent gatekeepers, heightening opportunities for scalability (i.e., ability of an intervention to reach adolescents and adolescent gatekeepers) and sustainability (i.e., cost-effectiveness). For the purposes of this review, technology-oriented is defined as use of Internet, webinar, social media, social network, blog, instant message, email, video, public service announcement, text message, teleconference, or telephone in delivering an intervention. To examine program impact, program efficacy, program reach, location of supporting program evidence, and quality of supporting research evidence are explored.

Method

Suicide prevention, adolescent, family, school, community, and emergency department programs contained within the National Registry of Evidence-Based Programs and Practices were reviewed (n = 127). Programs targeting adolescents or adolescent gatekeepers and including technology to address adolescent suicide prevention were included (n = 9): (1) Emergency Room (ER) Intervention for Adolescent Girls; (2) Family Intervention for Suicide Prevention (FISP); (3) Kognito At-Risk for High School Educators; (4) Lifelines Curriculum; (5) Linking Education and Awareness of Depression and Suicide (LEADS): For Youth; (6) Signs of Suicide (SOS); (7) Sources of Strength; (8) Strategies and Tools Embrace Prevention with Upstream Programs (STEP UP); and, (9) Question, Persuade, Refer (QPR). Registry reviewed studies with adolescent or adolescent gatekeeper samples were included, and additional studies evaluating these nine technology-oriented programs were searched.

Suicide prevention programs, practices, and informational materials contained within the Best Practices Registry also were reviewed (n = 114). Materials targeting adolescents or adolescent gatekeepers and including technology to address adolescent suicide prevention were included (n = 18): (1) ASK about Suicide to Save a Life; (2) Break Free From Depression: A 4-Session Curriculum Addressing Adolescent Depression; (3) Families Are Forever; (4) Friend2Friend; (5) Healthy Education for Life Program; (6) Helping Every Living Person Depression and Suicide Prevention Curriculum; (7) How Not To Keep A Secret; (8) LOOK, LISTEN, LINK; (9) Making Educators Partners in Youth Suicide Prevention: ACT on FACTS; (10) More Than Sad: Suicide Prevention Education for Teachers and School Personnel; (11) More Than Sad: Teen Depression; (12) Not My Kid: What Parents Should Know About Teen Suicide; (13) Plan, Prepare, Prevent: The SOS Online Gatekeeper Training; (14) Real Teenagers Talking About Depression: A Video-Based Study Guide; (15) Response: A Comprehensive High School-based Suicide Awareness Program (2nd ed.); (16) Step In, Speak Up!: Supporting LGBTQ Students; (17) Suicide Alertness for Everyone (safeTALK); (18) Suicide Prevention: A Gatekeeper Training for School Personnel. Studies evaluating these 18 technology-oriented registry materials, among adolescents and adolescent gatekeepers samples, were concurrently searched.

To identify studies evaluating technology-oriented programs contained in both registries, PubMed, PsycINFO, PsycARTICLES, and PASCAL information databases were systematically searched, as shown in the modified preferred reporting items for systematic reviews and meta-analyses (PRISMA) diagram (Moher et al. 2009) displayed in Fig. 1. Relevant names, titles, descriptors, and phrases relating to the population of interest (i.e., adolescents and adolescent gatekeepers), suicide and suicide prevention, technology, program titles, program developers, and program researchers were systematically searched to identify studies evaluating technology-oriented suicide prevention programs for adolescents and adolescent gatekeepers. Keywords relating to relevant names, titles, descriptors, and phrases, as well as specific Medical Subject Heading (meSH) terms, were searched with AND in combination with OR (Table 1). Original research was included. Reviews, reports, and books were excluded.

Fig. 1

Modified preferred reporting items for systematic reviews and meta-analyses (PRISMA) diagram

Table 1

Medical subject heading (meSH) terms and keywords

Overlapping/repeating meSH terms and keywords

Non-overlapping specific search words

Intervention; program; emergency room; emergency department; hospital; high school; school; school-based; secondary school; classroom; educators; enhanced; specialized; technology; kognito; avatar; documentary; videotape; video; DVD; telephone; mobile phone; calls; text message; SMS message; online; Internet; web; www; world wide web; virtual; blog; animated; human interaction game engine; virtual coach; simulation; email; electronic mail; instant message; social media; media; role play; curriculum; lecture; discussion; project; workshop; education; educate; awareness; teaching; teach; train; training; leader; knowledge; self-efficacy; recognize; recognition; identify; intervene; refer; respond; support; conversation; communication; interview; counseling; coping; survivor stories; peer; friend; classmate; student; teen; teenager; youth; adolescent; child; children; LGBT; lesbian; gay; bisexual; transgender; adult; parent; families; family; family member; family treatment; teacher; school personnel; gatekeeper; health care team; staff; crisis therapist; crisis support; therapy; follow-up; after care; outreach; outpatient; treatment adherence; motivation; motivate; commitment; continuity; coordinate; care; linkage; linking; link; connect; engage; interactive; empower; responsive; prevent; prevention; mental health; community; contacts; monitor; modify; manage; management; services; methods restriction; screening; screen; risk; at-risk; risk factors; warning signs; barriers; attitudes; norms; behavior; protective factors; safety; plan; triggers; suicidal ideation; suicide; suicide attempt; attempting suicide; suicide attempted; suicidal behavior; self harm; self-injurious behavior; help-seeking; seeking help; help; HELP; helping; sad; depression; anxiety; substance abuse; stigma; Kognito Interactive; Albright; Goldman; Washington Youth Suicide Prevention; Rauh; Society for the Prevention of Teen Suicide; Underwood; Screening for Mental Health, Inc.; and, American Foundation for Suicide Prevention

Females; girls; improved experience; approach; conceptualization; FISP; hope box; cognitive behavioral; psychological distress; LEADS; SOS; signs; ACT; acknowledge; tell; sources of strength; mentor; influences; presentation; posters; public service announcement; QPR; question; persuade; intent; diffusion; district; protocol; ASK; save; life; break; free; forever; gender identity; sexual orientation; sexual minority; Friend2Friend; healthy; life; living; person; secret; lifelines; look; listen; kid; plan; prepare; talking; study; guide; response; comprehensive; step in; speak up; alertness; everyone; safeTALK; QPR Institute; Haas; Azrak; Rotheram-Borus; Asarnow; LoMurray; SAVE; Leite; Mental Health America of Texas; Nudd; Boston Children’s Hospital; Jordan; Family Acceptance Project; Ryan; Church of Latter Day Saints; Heartline Oklahoma; Youth Health Connection; Green; Hazelden; Erika’s Lighthouse; ColumbiaCare; McConahay; Riverside Trauma Center; Bridger; Diamon; Swanner; and, LivingWorks

Published studies for six National Registry of Evidence-Based Programs and Practices interventions were identified in PubMed, PsycINFO, PsycARTICLES, and PASCAL information databases; however, no published studies evaluating Kognito At-Risk for High School Educators, Linking Education and Awareness of Depression and Suicide For Youth, and Strategies and Tools Embrace Prevention with Upstream Programs were identified. As a result, the search was expanded and CINAHL, Scopus, Cochrane Library, Google Scholar, and program organizational websites were searched. Two unpublished Kognito At-Risk for High School Educators studies were subsequently identified in Google Scholar. One unpublished Strategies and Tools Embrace Prevention with Upstream Programs study was also identified in Google Scholar. Alternatively, no studies evaluating Linking Education and Awareness of Depression and Suicide For Youth were identified, and the results of a study summarized in the national registry provided the only evidence for program synthesis.

Additionally, searches of PubMed, PsycINFO, PsycARTICLES, and PASCAL information databases resulted in the identification of published studies for 3 of the 18 Best Practices Registry programs: Making Educators Partners in Youth Suicide Prevention ACT on FACTS, Response A Comprehensive High School-based Suicide Awareness Program (2nd ed.), and Suicide Prevention A Gatekeeper Training for School Personnel. Consequently, the search was again expanded and CINAHL, Scopus, Cochrane Library, Google Scholar, and program organizational websites were searched. Studies evaluating the Suicide Alertness for Everyone program were subsequently identified on the program organizational website; however, no evidence for remaining programs and materials were identified. As a result, synthesis for the 14 remaining programs was not possible.

Results

In total, 35 published and unpublished studies were integrated to evaluate nine National Registry of Evidence-Based Programs and Practices interventions: Question, Persuade, Refer (n = 9); Signs of Suicide (n = 5); Kognito At-Risk for High School Educators (n = 4); Emergency Room Intervention for Adolescent Girls (n = 3); Lifelines Curriculum (n = 3); Sources of Strength (n = 2); Family Intervention for Suicide Prevention (n = 2); Linking Education and Awareness of Depression and Suicide For Youth (n = 1); and Strategies and Tools Embrace Prevention with Upstream Programs (n = 1); and to evaluate four Best Practices Registry interventions: Suicide Alertness for Everyone (n = 2); Making Educators Partners in Youth Suicide Prevention ACT on FACTS (n = 1); Response A Comprehensive High School-based Suicide Awareness Program (2nd ed., n = 1); and Suicide Prevention A Gatekeeper Training for School Personnel (n = 1).

National Registry of Evidence-Based Programs and Practices

Emergency Room Intervention for Adolescent Girls

For this intervention, emergency room staff receives training to enhance post-suicide care for adolescent girls and their families (Rotheram-Borus et al. 2000, 1996, 1999). The aim is to impact expectations of care and promote follow-up treatment adherence, as suicide vulnerability is elevated post-suicide attempt. During the emergency room visit, adolescent girls are shown a soap opera video that portrays adolescents impacted by suicide. The video explains emergency room procedures, coping strategies, and rationale for outpatient treatment. A crisis therapist discusses the video with girls and their families, providing a family treatment session in the emergency room. A crisis therapist also contracts with families to return for outpatient treatment (Rotheram-Borus et al. 2000, 1996, 1999).

The National Registry of Evidence-Based Programs and Practices overall quality of research ratings for studies evaluating this intervention were as follows (0.0–4.0 scale): treatment adherence 2.1, adolescent symptoms of depression 3.0, adolescent suicidal ideation 2.9, maternal symptoms of depression 2.8, and maternal attitudes toward treatment 2.2 (SAMHSA 2016). In a quasi-experimental study adolescents sequentially assigned to the intervention group reported significantly lower levels of depression and suicidal ideation post-discharge, compared to remaining adolescents sequentially assigned to standard care (n = 140, Rotheram-Borus et al. 1996). Adolescents were significantly more likely to attend outpatient treatment, attended non-significantly more treatment sessions, and were non-significantly less likely to drop out of treatment, than standard care adolescents (Rotheram-Borus et al. 1996). In a structural equations modeling assessment individual (i.e., age, psychopathology), structural (i.e., specialized vs. standard care), and family factors predicted treatment adherence, however, individual and structural factors were significantly greater predictors than family factors (n = 140, Rotheram-Borus et al. 1999). Correspondingly, specialized emergency room mothers were significantly less likely to attend all outpatient treatment sessions than standard care mothers (Rotheram-Borus et al. 1996). In a quasi-experimental comparative study evaluating outcomes at 18 months, suicide re-ideation and re-attempts were similar among specialized and standard care groups (n = 140, Rotheram-Borus et al. 2000). However, there were sustained significant reductions in depression among specialized emergency room adolescents and their mothers at 18 months (Rotheram-Borus et al. 2000).

Family Intervention for Suicide Prevention

An adaptation of the Emergency Room Intervention for Adolescent Girls, the Family Intervention for Suicide Prevention aims to enhance the emergency room experience for adolescents and their families to improve post-suicide care (Asarnow et al. 2011; Hughes and Asarnow 2013; SAMHSA 2016). The goal is to increase motivation for outpatient treatment, as suicide vulnerability is elevated post-suicide attempt. An emergency room family therapy session highlights the importance of outpatient treatment, methods restriction, and safety plans. Youth identify triggers of suicidal ideation, and create safety cards for future crises. Family support and creation of coping Hope Boxes are encouraged. Follow-up telephone calls begin 48-h post-discharge, continuing at 1-, 2-, 3-, and 4- weeks, or until adolescents are linked to outpatient care (Asarnow et al. 2011; Hughes and Asarnow 2013; SAMHSA 2016).

The National Registry of Evidence-Based Programs and Practices overall quality of research rating for the study evaluating this intervention was as follows (0.0–4.0 scale): linkage to outpatient mental health treatment services 3.1 (SAMHSA 2016). In a randomized controlled trial adolescents were significantly more likely to attend outpatient treatment, were significantly more likely to receive a greater number of psychotherapy sessions, were significantly more likely to receive combined psychotherapy and medication, and were significantly more likely to attend a greater number of outpatient visits, compared to standard care controls (n = 181, Asarnow et al. 2011). While there was no significant effect on suicide re-ideation or re-attempts, depression decreased significantly from baseline to 2 months, compared to controls (Asarnow et al. 2011). In an evaluation of the implementation and feasibility of the randomized controlled trial, the intervention was effectively delivered to 80.9% of adolescents in the emergency department (n = 181; Hughes and Asarnow 2013). Adolescents discharged prior to receiving the intervention later received the intervention on inpatient units (12.4%), community locations (3.4%), by phone (2.2%), or in an undocumented location (1.1%) (Hughes and Asarnow 2013). Most sessions included the adolescent and at least one parent (78.7%) (Hughes and Asarnow 2013). Finally, of the adolescents randomized to the intervention group (n = 89), 74.1% received the full intervention (Hughes and Asarnow 2013).

Kognito At-Risk for High School Educators

For this program, gatekeepers train individually online (Albright et al. 2011a, b, 2013; Ohio Suicide Prevention Foundation [OSPF] 2013; SAMHSA 2016). The self-paced, narrative-driven training includes videos and role-play conversations with fully animated and emotionally responsive student avatars. Communication strategies, methods for avoiding pitfalls in referral conversations, and real-time advice about connecting at-risk students to support resources are provided. Training goals include teaching school personnel to identify, approach, and refer students experiencing psychological distress (Albright et al. 2011a, b, 2013; OSPF 2013; SAMHSA 2016).

The National Registry of Evidence-Based Programs and Practices overall quality of research ratings for studies evaluating this intervention were as follows (0.0–4.0 scale): preparedness to recognize, approach, and refer psychologically distressed students 2.8; likelihood of approaching and referring psychologically distressed students 2.8; and confidence in one’s ability to help psychologically distressed students 2.8 (SAMHSA 2016). In a pre-experimental study (Albright et al 2011a), quasi-experimental study (Albright et al. 2011b), subsample analysis of a state-wide training initiative (OSFP 2013), and 3 month quasi-experimental longitudinal study (Albright et al. 2013), training significantly increased preparedness to recognize, approach, and refer at-risk students; significantly increased the likelihood of approaching and referring at-risk students; and significantly increased confidence in one’s ability to help suicidal students accept help, compared to controls (n = unknown, Albright et al. 2011a, b, 2013; n = unknown; n = 216; n = 303, OSFP 2013; SAMHSA 2016). At 3 months, total mental health skills remained significant, and teachers reported a 71% increase in approaching at-risk students (Albright et al. 2013).

Lifelines Curriculum

The goal of this school-based program is to enhance a school community’s ability to recognize and respond to troubled students (Haines 2007; Kalafat and Elias 1994; Kalafat et al. 2007; SAMHSA 2016). The program includes student curriculum, parent education, school personnel gatekeeper training, school crisis provider education, and administrative guidelines. The student curriculum utilizes video to depict peer suicide intervention steps. Video-related discussions and student role-play practice are also included (Haines 2007; Kalafat and Elias 1994; Kalafat et al. 2007; SAMHSA 2016). However, only three studies evaluating Lifelines Curriculum were identified, and in these studies only the student portion of the program was evaluated (Haines 2007; Kalafat and Elias 1994; Kalafat et al. 2007).

The National Registry of Evidence-Based Programs and Practices overall quality of research ratings for studies evaluating this intervention were as follows (0.0–4.0 scale): knowledge about suicide 2.9, attitudes about suicide and suicide intervention 2.9, attitudes about seeking adult help 2.9, and attitudes about keeping a friend’s suicide thoughts a secret 2.9 (SAMHSA 2016). In quasi-experimental studies (Haines 2007; Kalafat et al. 2007) and in a Solomon four-groups design study (Kalafat and Elias 1994) students experienced significant increases in knowledge about suicide, positive attitudes toward talking about suicide, positive attitudes toward suicide intervention, positive attitudes toward obtaining help for troubled peers, and positive attitudes toward seeking adult help, compared to controls (n = 86, Haines 2007; n = 253; Kalafat and Elias 1994; n = unknown; Kalafat et al. 2007; SAMHSA 2016).

Linking Education and Awareness of Depression and Suicide: For Youth

This school-based program utilizes teacher-led curriculum to increase knowledge of, teach symptoms of, demonstrate links between, modify perceptions of, and increase desirable attitudes toward depression and suicide (Leite et al. 2011; SAMHSA 2016). The intervention provides school and community resource information, and strategies to overcome help seeking barriers. Students experiencing depression and suicidal ideation are encouraged to seek help, on behalf of oneself as well as on behalf of troubled peers. Students work with and support their peers, while participating in awareness-raising projects and activities using simulated blogs, email, and instant messaging (Leite et al. 2011; SAMHSA 2016).

The National Registry of Evidence-Based Programs and Practices overall quality of research ratings for the study evaluating this intervention was as follows (0.0–4.0 scale): knowledge of depression and suicide 1.8, perceptions of depression and suicide 1.8, and knowledge of suicide prevention resources 1.8 (SAMHSA 2016). In a pre-experimental study, at post-test and at 3 month follow-up, students had significant increases in knowledge and perceptions of depression and suicide, and knowledge of suicide prevention resources, compared to control students (n = 730, Leite et al. 2011; SAMHSA 2016).

Signs of Suicide

This school-based program uses curriculum to increase awareness of depression and suicide, and includes a brief screen for depression and suicidal behavior (Aseltine 2003; Aseltine and DeMartino 2004; Aseltine et al. 2007; SAMHSA 2016; Schilling et al. 2014, 2016). The screen is self-scored with the aim of enhancing students’ ability to recognize personal risk. School resource information is provided, and students experiencing depression and suicidal ideation are encouraged to seek help. Students also learn to act on behalf of troubled classmates. Specifically, a video depicts signs of suicidal ideation and depression, and strategies to react to peers displaying signs. Discussion guides are used to facilitate video-related conversations. Further, as part of the school-based program, videos are also utilized to provide gatekeeper training to school personnel and parents (Aseltine 2003; Aseltine and DeMartino 2004; Aseltine et al. 2007; SAMHSA 2016; Schilling et al. 2014, 2016).

Signs of Suicide is one of the “Newly Reviewed” National Registry of Evidence-Based Programs and Practices interventions, and has been identified by the registry as producing “promising outcomes” for suicidal thoughts and behaviors, and knowledge, attitudes, and beliefs about mental health (SAMHSA 2016). However, the remaining two program outcomes, receipt of mental health and/or substance use treatment and social competence were identified by the registry as having “ineffective outcomes” (SAMHSA 2016). In randomized controlled trials students were 40% (n = 2100, Aseltine and DeMartino 2004; n = 4133; Aseltine et al. 2007) to 64% (n = 1046, Schilling et al. 2016) less likely to report a suicide attempt in the 3 months following intervention, compared to controls. In another randomized controlled trial, intervention students with pre-program suicidal ideation were 96% less likely to report suicidal behavior in the 3 months following intervention, compared to control students with pre-program ideation (n = 386, Schilling et al. 2014). Intervention students with suicide attempt history were 75% less likely to report suicidal planning in the 3 months following intervention, compared to control students with suicide attempt history (Schilling et al. 2016). Students experienced significantly greater knowledge of and more adaptive attitudes toward depression and suicide, compared to controls (Aseltine and DeMartino 2004; Aseltine et al. 2007; Schilling et al. 2016).

In a pre-experimental study, when compared to baseline, 30 days post-implementation there were 60% increases in personal help seeking for depression and suicidal ideation (n = 92 schools, Aseltine 2003). Compared to baseline, there were no increases in personal help seeking among intervention and control students at 3 months (Aseltine and DeMartino 2004; Aseltine et al. 2007). Compared to baseline, 30 days post-implementation, there were non-significant increases in seeking help on behalf of troubled friends (Aseltine 2003). Compared to baseline, there were no increases in helping troubled friends at 3 months, among intervention and control students (Aseltine and DeMartino 2004; Aseltine et al. 2007).

Sources of Strength

This school-based program trains student peer leaders to model positive behaviors and promote healthy coping practices, to modify peer norms and problem behaviors (Petrova et al. 2015; Wyman et al. 2010). Peer leaders promote eight protective sources of strength: family, positive friends, mentors, healthy activities, generosity, spirituality, medical access and mental health access. To disseminate protective sources of strength information, peer leaders send school-wide messages using social network sites, text messages, public service announcements, and videos. Peer leaders use technology to conduct positive-themed suicide prevention messaging activities, leveraging help from adult mentors and school personnel who are also trained. Peer leaders simultaneously encourage friends to name and engage trusted adults to increase youth-adult communication (Petrova et al. 2015; Wyman et al. 2010).

The National Registry of Evidence-Based Programs and Practices overall quality of research ratings for studies evaluating this intervention were as follows (0.0–4.0 scale): attitudes about seeking adult help for distress 3.1, knowledge of adult help for suicidal youth 3.1, rejection of codes of silence 3.1, referrals for distressed peers 3.0, and maladaptive coping attitudes 2.8 (SAMHSA 2016). In randomized controlled trials, when compared to untrained peer leaders, trained peer leaders reported significantly more positive expectations that adults at school should help suicidal peers, and peers should obtain adult help for suicidal friends despite peer requests for secrecy (n = 706, Petrova et al. 2015; n = 2675; Wyman et al. 2010). Compared to controls, help seeking norms and the number of identified trusted adults increased significantly (Petrova et al. 2015; Wyman et al. 2010), in addition to greater use of coping resources (Wyman et al. 2010). Trained peer leaders were significantly more engaged in school, increasing peer support, compared to untrained peer leaders (Wyman et al. 2010). Among students, perceptions of adult support for suicidal persons and acceptability of seeking help increased significantly, compared to controls (Petrova et al. 2015; Wyman et al. 2010). Students with past year suicidal ideation, relative to non-suicidal students, had greater perceptions of adult help (Petrova et al. 2015; Wyman et al. 2010), help-seeking acceptability, coping, and relationships with engaged adults (Petrova et al. 2015). Finally, trained peer leaders in metropolitan schools were 4-times more likely than untrained peer leaders to refer suicidal friends to engaged adults; however, referrals did not increase among trained peer leaders in smaller schools (Wyman et al. 2010).

Strategies and Tools Embrace Prevention with Upstream Programs

Before implementing this student curriculum, administrators attend an introductory webinar that provides education on staff training, structural issues, parental consent, and suicide prevention protocols (Fuller et al. 2015; SAMHSA 2016). Gatekeepers are also encouraged to participate in four teleconferences for additional instruction, and gatekeepers may request two additional teleconferences if needed, for a total of six teleconferences. The program is then implemented. The program includes social and emotional learning-based curriculum and activities for adolescents, with the goals of promoting positive mental health, enhancing emotional competence, and creating a safe school climate. To enhance positive learning experiences for students, components of the social-ecological model, social learning/social cognitive theory, metacognition, and mindfulness are included (Fuller et al. 2015; SAMHSA 2016).

Strategies and Tools Embrace Prevention with Upstream Programs is one of the “Newly Reviewed” National Registry of Evidence-Based Programs and Practices interventions, having been identified by the registry as producing “effective outcomes” for self-regulation and social competence (SAMHSA 2016). In an experimental study employing a pre- and post- design, students randomized to the program had significant gains in teacher-rated social and emotional learning, compared to control group students (n = 59, Fuller et al. 2015). Specifically, teachers reported that students exhibited significantly greater self-regulation, social competence, and responsibility, compared to control students (Fuller et al. 2015).

Question, Persuade, Refer

For this program, gatekeepers have the option to train individually online (SAMHSA 2016). Both online and in-person training also utilize videos, which feature individuals and families impacted by suicide (SAMHSA 2016). Training highlights suicide epidemiology, myths, facts, statistics, warning signs, communication strategies, and resource information, with the goal of developing three gatekeeper skills: Question the individual’s suicidal desire or intent, Persuade the individual to seek and accept help, and Refer the individual to resources (Cerel et al. 2012; Coleman and Del Quest 2015; Cross et al. 2011; Johnson and Parsons 2012; Keller et al. 2009; Reis and Cornell 2008; SAMHSA 2016; Tompkins et al. 2009, 2010; Wyman et al. 2008).

The National Registry of Evidence-Based Programs and Practices overall quality of research ratings for studies evaluating this intervention were as follows (0.0–4.0 scale): knowledge about suicide 2.6, gatekeeper self-efficacy 2.6, knowledge of suicide prevention resources 2.9, gatekeeper skills 2.8, and diffusion of gatekeeper training information 2.5 (SAMHSA 2016). In state-wide training initiatives (Cerel et al. 2012; Keller et al. 2009), quasi-experimental studies (Coleman and Del Quest 2015; Reis and Cornell 2008; Tompkins et al. 2009, 2010), a comparative experimental study with 3 month follow-up (Cross et al. 2011), a randomized controlled trial with 12 month follow-up (Wyman et al. 2008), and a quality improvement initiative (Johnson and Parsons 2012), training significantly increased self-perceived knowledge and self-efficacy (n = 3958, Cerel et al. 2012; n = 126; Coleman and Del Quest 2015; n = 170; Cross et al. 2011; n = 36; Johnson and Parsons 2012; n = 630; Keller et al. 2009; n = 238; Reis and Cornell 2008; n = 106; Tompkins et al. 2009, 2010; n = 106; n = 249; Wyman et al. 2008); significantly increased declarative knowledge (Cross et al. 2011; Johnson and Parsons 2012; Reis and Cornell 2008; Tompkins et al. 2009, 2010), knowledge of access to services (Johnson and Parsons 2012; Wyman et al. 2008), asking students about suicide (Coleman and Del Quest 2015; Cross et al. 2011; Johnson and Parsons 2012; Reis and Cornell 2008; Tompkins et al. 2009, 2010; Wyman et al. 2008), and self-reported referrals of students by school personnel (Coleman and Del Quest 2015; Cross et al. 2011; Reis and Cornell 2008).

Best Practices Registry

Making Educators Partners in Youth Suicide Prevention: ACT on FACTS

This training addresses school personnel’s role in identifying and referring at-risk youth. For this program, gatekeepers train individually online (Lamis et al. 2016; SPRC 2016). Videos of families impacted by suicide, recorded expert dialogue, and interactive role-plays are also included. Suicide myths, prevalence, warning signs, risk and protective factors, educator roles, school responsibilities, interaction strategies, resource information, and referral processes are highlighted. The goal is to increase awareness and understanding of youth suicide, while building competent school communities. (Lamis et al. 2016; SPRC 2016).

The Best Practices Registry does not evaluate the quality of research of the following study evaluating this intervention. In a pre-experimental study training was associated with significant increases in knowledge, self-efficacy, confidence, and attitudes among school personnel gatekeepers, and training satisfaction was high (n = 700; Lamis et al. 2016). Teachers and classroom aids in particular experienced large increases in self-efficacy and confidence, compared to guidance counselors and administrators (Lamis et al. 2016).

Response: A Comprehensive High School-based Suicide Awareness Program (2nd ed.)

This school-based program includes youth curriculum, school personnel gatekeeper training, advanced training for select staff (e.g., counselors), and informational materials for parents (Coleman and Del Quest 2015; SPRC 2016). The school resource kit includes video, and videos supplement the suicide awareness curriculum. Program goals include increasing knowledge of depression and suicide, knowledge of help-seeking barriers, referral steps, resources, and referrals of at-risk youth (Coleman and Del Quest 2015; SPRC 2016). However, only one study evaluating the intervention was identified, and in this study only the gatekeeper portion of the program was evaluated (Coleman and Del Quest 2015).

The Best Practices Registry does not evaluate the quality of research of the following study evaluating this intervention. In a quasi-experimental study employing a three-group repeated measures design, training was associated with significantly increased gatekeeper suicide prevention preparedness, attitudes, efficacy, asking students about suicide, and self-reported referrals of at-risk youth (n = 126; Coleman and Del Quest 2015).

Suicide Alertness for Everyone

Training goals include preparing gatekeepers to identify and connect suicidal individuals to professional resources (McLean et al. 2007; Niagara Suicide Prevention Coalition [NSPC] 2015). Gatekeepers are taught to identify direct and indirect invitations for help, recognize warning signs, and engage individuals requiring help. Specifically, videos illustrate non-alert and alert responses to direct and indirect requests for help. Strategies to overcome reactions to avoid suicidal individuals are also included, by applying the TALK steps: Tell, Ask, Listen, and Keep Safe (McLean et al. 2007; NSPC 2015).

The Best Practices Registry does not evaluate the quality of research of the following studies evaluating this intervention. In pre-experimental studies training was associated with improved gatekeeper beliefs, attitudes, knowledge, confidence and preparedness, and training satisfaction was high (n = 239, McLean et al. 2007; n = 317, NSPC 2015). At 3 months, over 90% of participants believed training was useful (n = 34; NSPC 2015). At 6 months, 24% of respondents reported using their gatekeeper skills to help someone at risk of suicide (n = 34; McLean et al. 2007).

Suicide Prevention: A Gatekeeper Training for School Personnel

Training provides education on suicide assessment and management (Mirick et al. 2016; SPRC 2016). This program includes 12 modules that review suicide facts, protective factors, risk factors, warning signs, managing reactions, therapeutic empathy, communication strategies, risk formulation, and intervention. Training also includes reviews of safety planning, treatment practices, standard of care, postvention, self-care, and includes stories of hope. Videos supplement the curriculum by supporting discussions and role-play practice (Mirick et al. 2016; SPRC 2016).

The Best Practices Registry does not evaluate the quality of research of the following study evaluating this intervention. In a pre-experimental study training was associated with significantly increased knowledge and confidence in suicide assessment and intervention (n = 442; Mirick et al. 2016). Mental health and non-mental health professionals completed training; mental health professionals had higher posttest scores than non-mental health professionals. Mental health professionals indicated safety planning, validity techniques, assessment tools, and strategies for asking about suicidal ideation/behavior were the most beneficial aspects of training (Mirick et al. 2016).

Discussion

Technology-oriented suicide prevention programs for adolescents and adolescent gatekeepers included in the National Registry of Evidence-Based Programs and Practices and Best Practices Registry were examined to explore program efficacy, program reach, location of supporting program evidence, and quality of supporting research evidence. Findings are summarized in each of these respective domains.

Program Efficacy

Several interventions were identified as efficacious, improving secondary outcomes, suicide outcomes, and gatekeeper preparedness. Programs led to reductions in suicide attempts (Aseltine and DeMartino 2004; Aseltine et al. 2007; Schilling et al. 2016), and yielded post-program reductions in suicidal planning and suicidal behaviors (Petrova et al. 2015; Schilling et al. 2014, 2016; Wyman et al. 2010). Knowledge and adaptive attitudes toward depression and suicide improved (Aseltine and DeMartino 2004; Aseltine et al. 2007; Haines 2007; Kalafat and Elias 1994; Kalafat et al. 2007; Leite et al. 2011; Schilling et al. 2016), as did knowledge of suicide prevention and coping resources (Haines 2007; Kalafat and Elias 1994; Kalafat et al. 2007; Leite et al. 2011; Petrova et al. 2015; Wyman et al. 2010). Acceptability of seeking help (Haines 2007; Kalafat and Elias 1994; Kalafat et al. 2007; Petrova et al. 2015; Wyman et al. 2010), and social regulation and self-competence (Fuller et al. 2015) also increased.

Adolescents with a recent suicide attempt also appeared to benefit from emergency room interventions. Youth had increased likelihood of receiving outpatient treatment, and receipt of a greater number of outpatient therapy sessions (Asarnow et al. 2011; Hughes and Asarnow 2013; Rotheram-Borus et al. 1996). While analyses did not reveal sustained effects relative to suicidal ideation and suicide attempts, there were sustained reductions in depressive symptoms (Asarnow et al. 2011; Rotheram-Borus et al. 2000). Finally, programs increased adolescent gatekeeper suicide prevention preparedness, with training diffusion ultimately benefiting adolescents (Albright et al. 2011a, b, 2013; Cerel et al. 2012; Coleman and Del Quest 2015; Cross et al. 2011; Johnson and Parsons 2012; Keller et al. 2009; Lamis et al. 2016; McLean et al. 2007; Mirick et al. 2016; NSPC 2015; OSPF 2013; Reis and Cornell 2008; Tompkins et al. 2009, 2010; Wyman et al. 2008).

Program Reach

Numerous interventions demonstrated potential to reach the target population. Potential for reach is high given the scalability and sustainability of most programs. Programs that solely leveraged Internet resources (i.e., Linking Education and Awareness of Depression and Suicide For Youth) had particularly high potential for scalability given the accessibility and reach of online resources. Additionally, programs that solely leveraged video (i.e., Emergency Room Intervention for Adolescent Girls; Lifelines Curriculum; Response A Comprehensive High School-based Suicide Awareness Program (2nd ed.); Suicide Alertness for Everyone; Signs of Suicide; Suicide Prevention A Gatekeeper Training for School Personnel) and/or that solely leveraged telephone (i.e., Family Intervention for Suicide Prevention) had elevated potential for sustainability given the low cost of video and telephone outreach. It is unclear how these programs compare to programs that integrate multiple technology platforms and educational approaches concurrently (i.e., Kognito At-Risk for High School Educators; Making Educators Partners in Youth Suicide Prevention ACT on FACTS; Sources of Strength; Strategies and Tools Embrace Prevention with Upstream Programs; Question, Persuade, Refer), because few studies have conducted head-to-head comparisons. It is possible that some approaches have greater efficacy, but lower reach than wholly technology-based interventions due to cost and resources associated with implementing widely. Research is needed to draw these comparisons and fully elucidate the cost effectiveness of these programs.

Although potential for reach is high due to the focus on technology-oriented resources, some of the most highly accessible programs have limited research evidence. In studies including adolescents or adolescent gatekeepers, Question, Persuade, Refer contained numerous studies with supporting evidence (n = 9), whereas Linking Education and Awareness of Depression and Suicide For Youth (n = 1), Strategies and Tools Embrace Prevention with Upstream Programs (n = 1), Making Educators Partners in Youth Suicide Prevention ACT on FACTS (n = 1), Response A Comprehensive High School-based Suicide Awareness Program (2nd ed., n = 1), and Suicide Prevention A Gatekeeper Training for School Personnel (n = 1) were each evaluated in only one study. Many interventions listed in the Best Practices Registry do not have evaluation data in the published literature. Future research should be prioritized, to determine which interventions are most effective and to build the evidence base, promoting the dissemination and reach of programs that are supported by substantial research evidence.

An encouraging finding in this review was that several studies were conducted with large, population-based samples, representing statewide suicide prevention initiatives (Cerel et al. 2012; Keller et al. 2009; OSFP 2013), and demonstrating the reach of programs nationally. Some programs have also been implemented internationally (SAMHSA 2016; SPRC 2016), demonstrating the potential reach and impact of programs globally. These include the Emergency Room Intervention for Adolescent Girls; Linking Education and Awareness of Depression and Suicide For Youth; Suicide Alertness for Everyone; Signs of Suicide; and Question, Persuade, Refer programs. Additional large-scale research initiatives would promote the dissemination of well-researched programs, and are also needed to develop a more comprehensive understanding of reach and population impact.

Location of Supporting Program Evidence

The accessibility of supporting evidence varied, and was often reduced through dispersion. Among National Registry of Evidence-Based Programs and Practices interventions, Kognito At-Risk for High School Educators, Lifelines Curriculum, Linking Education and Awareness of Depression and Suicide For Youth, and Strategies and Tools Embrace Prevention with Upstream Programs ratings were based on studies not published in the literature (Albright et al. 2011a, b; Fuller et al. 2015; Kalafat et al. 2007; Leite et al. 2011). In contrast, ratings and study replications for remaining programs (i.e., Emergency Room Intervention for Adolescent Girls; Family Intervention for Suicide Prevention; Signs of Suicide; Sources of Strength; Question, Persuade, Refer) arose from published studies contained in the literature (Asarnow et al. 2011; Aseltine 2003; Aseltine and DeMartino 2004; Aseltine et al. 2007; Rotheram-Borus et al. 2000, 1996; Cross et al. 2011; Hughes and Asarnow 2013; Keller et al. 2009; Reis and Cornell 2008; Tompkins et al. 2009, 2010; Wyman et al. 2008, 2010). However, some evidence was found in Google Scholar (i.e., Kognito At-Risk for High School Educators), and was not identified in the national registry (Albright et al. 2013; OFSP 2013). Further, other evidence was found in information databases and contained in the literature, but was not identified in or evaluated by the national registry (Cerel et al. 2012; Coleman and Del Quest 2015; Haines 2007; Johnson and Parsons 2012; Kalafat and Elias 1994; Petrova et al. 2015; Rotheram et al. 1999; Schilling et al. 2014, 2016). Among Best Practices Registry interventions, published preliminary evaluations were identified in the medical literature (Coleman and Del Quest 2015; Lamis et al. 2016; Mirick et al. 2016), and unpublished preliminary evaluations were identified on a program organizational website (McLean et al. 2007; NSPC 2015).

Because supporting program evidence was highly dispersed, locating evidence was sometimes challenging. The national registries intentionally target diverse stakeholders, however, evidence supporting program dissemination and implementation is not always readily accessible among intended stakeholders. If stakeholders and the target population were increasingly able to identify the settings, samples, program procedures, and outcomes in which programs were implemented and evaluated, opportunities for replication and implementation may increase. Specifically, aggregating and improving accessibility of supporting information may enhance dissemination and reach, while also partially ensuring optimal implementation, to enhance program impact.

Quality of Supporting Research Evidence

Regardless of the location and accessibility of evidence, supporting evidence was generally strong. Among studies including adolescents and adolescent gatekeepers, the Family Intervention for Suicide Prevention (Asarnow et al. 2011; Hughes and Asarnow 2013), Signs of Suicide (Aseltine and DeMartino 2004; Aseltine et al. 2007; Schilling et al. 2016, 2014), Sources of Strength (Petrova et al. 2015; Wyman et al. 2010), Strategies and Tools Embrace Prevention with Upstream Programs (Fuller et al. 2015), and Question, Persuade, Refer (Cross et al. 2011; Wyman et al. 2008) programs are the only interventions evaluated in randomized controlled trials. Further, the Strategies and Tools Embrace Prevention with Upstream Programs intervention is currently one of only three programs in the National Registry of Evidence-Based Programs and Practices recognized as having “effective” outcomes (SAMHSA 2016), creating an opportunity to review and evaluate remaining programs to establish or highlight their ability to produce effective outcomes. More rigorous study methods and large samples are needed to increase quality of research ratings, which may promote the dissemination of rigorously evaluated programs with highly effective outcomes.

Because the majority of Best Practices Registry programs have not yet been evaluated, researchers also have opportunities to establish the efficacy of these interventions. Preliminarily evaluated programs currently listed in the Best Practices Registry also would benefit from additional evaluation. Increasing program availability would increasingly enable diverse stakeholders and the target population to select a program that best fit their particular needs, as well as the unique demands or constraints of the setting in which the program would be implemented, to enhance impact. Additionally, while the majority of programs have been evaluated within the last decade, the Emergency Room Intervention for Adolescent Girls (Rotheram-Borus et al. 1996, 1999, 2000) and Lifelines Curriculum (Haines 2007; Kalafat and Elias 1994; Kalafat et al. 2007) have not been evaluated for 10 or more years. As technology continues to evolve and social, political, economic, environmental climates change, it is important that timely quality studies continue to keep the evidence base up to date.

Further, a relatively limited number of published studies were identified in the medical literature (n = 26 National Registry of Evidence-Based Programs and Practices; n = 3 Best Practices Registry). It is important to consider the quality of both published and unpublished evidence. Published studies in the medical literature undergo rigorous peer review, but the quality of the research methodology is not always superior to unpublished studies. Unpublished studies independently reviewed by the Substance Abuse and Mental Health Services Administration and included in the National Registry of Evidence-Based Programs and Practices, for example, undergo rigorous review. However, because published reports tend to detail study methods as well as the guiding theoretical framework, statistical analyses, results, and conclusions, evaluation of quality generally is more feasible. Gauging the quality of unpublished studies is sometimes more difficult, as one must rely on a study abstract or study summary only in some cases. Greater opportunities for cost-effective program implementation are possible through increased access to high quality, rigorous research necessary to guiding decision making.

Conclusion

Technology-oriented suicide prevention programs for adolescents and adolescent gatekeepers included in the National Registry of Evidence-Based Programs and Practices and Best Practices Registry were examined to explore program efficacy, program reach, accessibility of supporting program evidence, and quality of supporting research evidence. Many interventions were identified as efficacious, producing positive outcomes related to youth suicide prevention and gatekeeper preparedness. Program reach was more variable. Technology generally promoted the reach of prevention content among adolescents and adolescent gatekeepers, suggesting high scalability and cost efficiency. The sustainability of programs also appears promising given the low cost of included technology components. Consistent with this notion, several studies contained large national and international samples.

The location and accessibility of supporting program evidence varied. Evidence was often highly dispersed among several information forums, which limited the accessibility of supporting evidence. Although the national registries remain committed to centrally locating supporting evidence, variability is high. Nevertheless, the quality of supporting research evidence was generally strong, with the majority of studies transparently including clear details regarding study methodology. However, some study reports were less transparent, and summaries were the only information provided. National Registry of Evidence-Based Programs and Practices quality of research ratings were generally strong, although there are opportunities to evaluate programs using more rigorous methodologies and larger adolescent and adolescent gatekeeper samples. There are also opportunities to evaluate Best Practices Registry programs using increasingly rigorous study methods.

Additional research is needed to strengthen the evidence base. Readily accessible research evidence is needed to guide decision-making, and to improve reach and sustained use of efficacious, scalable programs. Adolescent suicide remains a major public health priority, and this review reveals that it is critical that effective programs become more widely available to adolescents and adolescent gatekeepers, with clear evidence-based policies in place.

Notes

Acknowledgements

We would like to thank Lisa Kerr, Ph.D. for providing editorial feedback and APA formatting assistance.

Author Contributions

Both authors were responsible for the conception and direction of the article. EK created the first draft of the article, and KJR provided substantive feedback on subsequent drafts. After several iterations where both authors contributed, both authors approved the final version of the article.

Compliance with Ethical Standards

Conflict of interest

The authors report no conflicts of interest.

Funding

There were no forms of financial support, funding, or involvement.

References

  1. Albright, G., Eastgard, S. Goldman, R., & Shockley, K. (2011a). At-risk for high school educators: On-line interactive gatekeeper training simulation for identification and referral of students exhibiting signs of psychological distress. http://legacy.nreppadmin.net/ViewIntervention.aspx?id=317. Accessed 26 Mar 2017 (unpublished manuscript).
  2. Albright, G., Eastgard, S. Goldman, R., & Shockley, K. (2011b). At-risk for high school educators: On-line interactive gatekeeper training simulation for identification and referral of students exhibiting signs of psychological distress. http://legacy.nreppadmin.net/ViewIntervention.aspx?id=317. Accessed 26 Mar 2017 (unpublished manuscript).
  3. Albright, G., Goldman, R., Shockley, K., & Spiegler, J. A. (2013). Using simulated conversations with virtual humans to build mental health skills among high school educators: A longitudinal study. http://resources.kognito.com/ht/kognito_study_highschool_version.pdf. Accessed 26 Mar 2017 (unpublished manuscript).
  4. Asarnow, J. R., Baraff, L., Berk, M., Grob, C., Devich-Navarro, M., Suddath, R., Piacentini, J., Rotheram-Borus, M. J., Cohen, D., & Tang, L. (2011). Effects of an emergency department mental health intervention for linking pediatric suicidal patients to follow-up mental health treatment: A randomized controlled trial. Psychiatric Services, 62(11), 1303–1309. doi: 10.1176/appi.ps.62.11.1303.CrossRefPubMedPubMedCentralGoogle Scholar
  5. Aseltine, R. H. (2003). An evaluation of a school based suicide prevention program. Adolescent and Family Health, 3(2), 81–88.Google Scholar
  6. Aseltine, R. H., & DeMartino, R. (2004). An outcome evaluation of the SOS suicide prevention program. American Journal of Public Health, 94(3), 446–451.CrossRefPubMedPubMedCentralGoogle Scholar
  7. Aseltine, R. H., James, A., Schilling, E. A., & Glanovsky, J. (2007). Evaluating the SOS suicide prevention program: A replication and extension. BioMed Central. Public health, 7, 161. doi: 10.1186/1471-2458-7-161.PubMedPubMedCentralGoogle Scholar
  8. Centers for Disease Control and Prevention. (2015). Suicide prevention: Youth suicide. http://www.cdc.gov/violenceprevention/suicide/youth_suicide.html. Accessed 22 June 2016.
  9. Centers for Disease Control and Prevention. (2016a). CDC grand rounds: Preventing suicide through a comprehensive public health approach. MMWR Report prepared by Corinne David-Ferdon, Alex E. Crosby, Eric D. Caine, Jarrod Hindman, Jerry Reed, and John Iskander. https://www.cdc.gov/mmwr/volumes/65/wr/mm6534a2.htm. Accessed 11 Aug 2016.
  10. Centers for Disease Control and Prevention. (2016b). Children’s mental health. http://www.cdc.gov/childrensmentalhealth/. Accessed 21 June 2016.
  11. Centers for Disease Control and Prevention. (2016c). Increase in suicide rates in the United States, 1999–2014. NCHS Data Brief No. 241 prepared by Sally C. Curtin, Margaret Warner, and Holly Hedegaard. http://www.cdc.gov/nchs/data/databriefs/db241.pdf.
  12. Cerel, J., Padgett, J. H., Robbins, V., & Kaminer, B. (2012). A state’s approach to suicide prevention awareness: Gatekeeper training in Kentucky. Journal of Evidence-Based Social Work, 9(3), 283–292. doi: 10.1080/15433714.2010.498672.CrossRefPubMedGoogle Scholar
  13. Coleman, D., & Del Quest, A. (2015). Science from evaluation: Testing hypotheses about differential effects of three youth-focused suicide prevention trainings. Social Work in Public Health, 30(2), 117–128. doi: 10.1080/19371918.2014.938397.CrossRefPubMedGoogle Scholar
  14. Cross, W. F., Seaburn, D., Gibbs, D., Schmeelk-Cone, K., White, A. M., & Caine, E. D. (2011). Does practice make perfect? A randomized control trial of behavioral rehearsal on suicide prevention gatekeeper skills. Journal of Primary Prevention, 32(3–4), 195–211. doi: 10.1007/s10935-011-0250-z.CrossRefPubMedPubMedCentralGoogle Scholar
  15. Fuller, M.A., Haboush-Deloye, A., Goldberg, P., & Grob, K. (2015). Strategies & tools to embrace prevention w/ upstream programs (STEP UP): A comprehensive evaluation report. Nevada Institute for Children’s Research and Policy, School of Community Health Services, University of Nevada, Las Vegas. http://nic.unlv.edu/files/STEP%20UP%20Comprehensive%20Report_FINAL.pdf. Accessed 20 Mar 2017.
  16. Haines, C. V. (2007). An evaluation of the proximal outcomes of a school based suicide awareness program: Changes in knowledge, attitudes, and responses to analog scenarios. Ann Arbor, MI: ProQuest Information and Learning Company.Google Scholar
  17. Hughes, J. L., & Asarnow, J. R. (2013). Enhanced mental health interventions in the emergency department: Suicide and suicide attempt prevention in the ED. Clinical Pediatric Emergency Medicine, 14(1), 28–34. doi: 10.1016/j.cpem.2013.01.002.CrossRefPubMedPubMedCentralGoogle Scholar
  18. Johnson, L.A., & Parsons, M.E. (2012). Adolescent suicide prevention in a school setting: Use of a gatekeeper program. NASN School Nurse, 27(6), 312–317. doi: 10.1177/1942602X12454459.CrossRefPubMedGoogle Scholar
  19. Kalafat, J., & Elias, M. (1994). An evaluation of a school-based suicide awareness intervention. Suicide and Life-Threatening Behavior, 24(3), 224–233.PubMedGoogle Scholar
  20. Kalafat, J., Madden, M., Haley, D., & O’Halloran, S. (2007). Evaluation of Lifelines classes: A component of the school-community based Maine youth suicide prevention project. http://legacy.nreppadmin.net/ViewIntervention.aspx?id=37. Accessed 24 Mar 2017 (unpublished manuscript).
  21. Keller, D. P., Schut, L. J., Puddy, R. W., Williams, L., Stephens, R. L., McKeon, R., & Lubell, K. (2009). Tennessee lives count: Statewide gatekeeper training for youth suicide prevention. Professional Psychology: Research and Practice, 40(2), 126–133. doi: 10.1037/a0014889.CrossRefGoogle Scholar
  22. Lamis, D. A., Underwood, M., & D’Amore, N. (2016). Outcomes of a suicide prevention gatekeeper training program among school personnel. Crisis: The Journal of Crisis Intervention and Suicide Prevention, 38(2), 89–99. doi: 10.1027/0227-5910/a000414.CrossRefGoogle Scholar
  23. Leite, A., Idzelis, M., Reidenberg, D., Roggenbaum, S., & LeBlanc, A. (2011). Linking Education and Awareness of Depression and Suicide (LEADS): An evaluation of a school-based suicide prevention curriculum for high school youth. St. Paul, MN: Wilder Research. http://legacy.nreppadmin.net/ViewIntervention.aspx?id=269. Accessed 24 Mar 2017.
  24. McLean, J., Schinkel, M., Woodhouse, A., Pynnonen, A. M., & McBryde, L. (2007). Evaluation of the Scottish safeTALK pilot. https://www.livingworks.net/resources-and-support/research-and-evaluation/. Accessed 26 Mar 2017.
  25. Mirick, R., McCauley, J., Bridger, J., & Berkowitz, L. (2016). Continuing education on suicide assessment and crisis intervention: What can we learn about the needs of mental health professionals in community practice. Community Mental Health Journal, 52(5), 501–510. doi: 10.1007/s10597-015-9884-2.CrossRefPubMedGoogle Scholar
  26. Moher, D., Liberati, A., Tetzlaff, J., Altman, D.G., The PRISMA Group (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med, 6(7), e1000097. doi: 10.1371/journal.pmed1000097. Diagram retrieved from http://www.prisma-statement.org. Accessed 30 May 2016.
  27. National Action Alliance for Suicide Prevention. (2014). A prioritized research agenda for suicide prevention: An action plan to save lives. Research Prioritization Task Force; National Strategy for Suicide Prevention. http://actionallianceforsuicideprevention.org/sites/actionallianceforsuicideprevention.org/files/Agenda.pdf.
  28. National Alliance on Mental Illness. (2016). Mental health facts: Children and teens. https://www.nami.org/getattachment/Learn-More/Mental-Health-by-the-Numbers/childrenmhfacts.pdf. Accessed 28 July 2016.
  29. Niagara Suicide Prevention Coalition (2015). Evaluation of safeTALK training in a convenience sample of 500 Niagara health region residents, health professionals, and volunteers, 2015. http://niagarasuicideprevention.weebly.com/uploads/6/9/5/2/6952634/safetalk-report-2014-final2.pdf. Accessed 26 Mar 2017.
  30. Ohio Suicide Prevention Foundation. (2013). Ohio’s campaign for hope: Youth suicide prevention initiative. http://www.ohiospf.org/files/OSPFAnnualReportYear2SAMHSAGrant5U79SM060428-02.pdf. Accessed 26 Mar 2017.
  31. Petrova, M., Wyman, P. A., Schmeelk-Cone, K., & Pisani, A. R. (2015). Positive-themed suicide prevention messages delivered by adolescent peer leaders: Proximal impact on classmates’ coping attitudes and perceptions of adult support. Suicide and Life-Threatening Behavior, 45(6), 651–663. doi: 10.1111/sltb.12156.CrossRefPubMedPubMedCentralGoogle Scholar
  32. Reis, C., & Cornell, D. (2008). An evaluation of suicide gatekeeper training for school counselors and teachers. Professional School Counseling, 11(6), 386–394. doi: 10.5330/PSC.n.2010-11.386.CrossRefGoogle Scholar
  33. Rotheram-Borus, M. J., Piacentini, J., Van Rossem, R., Graae, F., Cantwell, C., Castro-Blanco, D., Miller, S., & Feldman, J. (2000). The 18-month impact of an emergency room intervention for adolescent female suicide attempters. Journal of Consulting and Clinical Psychology, 68(6), 1081–1093. doi: 10.1037//0022-006X.68.6.1081.CrossRefPubMedGoogle Scholar
  34. Rotheram-Borus, M. J., Piacentini, J., Van Rossem, R., Graae, F., Cantwell, C., Castro-Blanco, D., Feldman, J. (1996). Enhancing treatment adherence with a specialized emergency room program for adolescent suicide attempters. American Academy of Child and Adolescent Psychiatry, 35(5), 654–663. doi: 10.1097/00004583-199605000-00021.CrossRefGoogle Scholar
  35. Rotheram-Borus, M. J., Piacentini, J., Van Rossem, R., Graae, F., Cantwell, C., Castro-Blanco, D., & Fledman, J. (1999). Treatment adherence among Latina female adolescent suicide attempters. Suicide and Life-Threatening Behavior, 29(4), 319–331. doi: 10.1111/j.1943-278X.1999.tb00527.x.PubMedGoogle Scholar
  36. Schilling, E. A., Aseltine, R. H., & James, A. (2016). The SOS suicide prevention program: Further evidence of efficacy and effectiveness. Prevention Science, 17(2), 157–166. doi: 10.1007/s11121-015-0594-3.CrossRefPubMedGoogle Scholar
  37. Schilling, E. A., Lawless, M., Buchanan, L., & Aseltine, R. H. (2014). “Signs of Suicide” shows promise as a middle school suicide prevention program. Suicide and Life-Threatening Behavior, 44(6), 653–667. doi: 10.1111/sltb.12097.CrossRefPubMedGoogle Scholar
  38. Substance Abuse and Mental Health Services Administration. (2016). National registry of evidence-based programs and practices. http://nrepp.samhsa.gov/AdvancedSearch.aspx. Accessed 20 Mar 2017.
  39. Suicide Prevention Resource Center. (2016). Best practices registry. http://www.sprc.org/bpr/section-iii-adherence-standards. Accessed 20 Mar 2017.
  40. Tompkins, T. L., Witt, J., & Abraibesh, N. (2009). Does a gatekeeper suicide prevention program work in a school setting? Evaluating training outcome and moderators of effectiveness. Suicide and Life-Threatening Behavior, 39(6), 671–681.CrossRefPubMedGoogle Scholar
  41. Tompkins, T. L., Witt, J., & Abraibesh, N. (2010). Does a gatekeeper suicide prevention program work in a school setting? Evaluating training outcome and moderators of effectiveness. Suicide and Life-Threatening Behavior, 40(5), 506–515. doi: 10.1521/suli.2010.40.5.506.CrossRefPubMedGoogle Scholar
  42. Wyman, P. A., Brown, C. H., Inman, J., Cross, J., Schmeelk-Cone, K., Guo, J., & Pena, J. B. (2008). Randomized trial of a gatekeeper program for suicide prevention: 1-year impact on secondary school staff. Journal of Consulting and Clinical Psychology, 76(1), 104–115. doi: 10.1037/0022-006X.76.1.104.CrossRefPubMedPubMedCentralGoogle Scholar
  43. Wyman, P. A., Brown, C. H., LoMurray, M., Schmeelk-Cone, K., Petrova, M., Yu, Q., Walsh, E., Tu, X., & Wang, W. (2010). An outcome evaluation of the sources of strength suicide prevention program delivered by adolescent peer leaders in high schools. American Journal of Public Health, 100(9), 1653–1661. doi: 10.2105/AJPH.2009.190025.CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Springer International Publishing 2017

Authors and Affiliations

  1. 1.Medical University of South CarolinaCollege of NursingCharlestonUSA
  2. 2.Ralph H. Johnson VA Medical CenterCharlestonUSA

Personalised recommendations