Testing for Reliability of the TARGET Threat Analysis Instrument (TTAI): An Interdisciplinary Instrument for the Analysis of School Shooting Threats

  • Nadine AhligEmail author
  • Kristin Göbel
  • Mirko Allwinn
  • Nora Fiedler
  • Vincenz Leuschner
  • Herbert Scheithauer
Part of the Security Informatics and Law Enforcement book series (SILE)


This chapter describes the development and interrater reliability analysis for a standardised research tool to analyse the characteristics of school shooting threats systematically, based on state-of-the-art knowledge. The instrument was developed on the basis of the current but not empirically tested approaches evaluating the seriousness of school shooting threats. An interrater reliability study was conducted following two rating phases and instrument revisions with 13 independent raters evaluating school shooting threat case records (N = 15). Most items showed high reliability after final modifications (90%; N = 88). The TARGET Threat Analysis Instrument (TTAI) is a reliable tool for testing current approaches and developing elaborated criteria to distinguish between school shooting threats which are meant to be serious and threats which are situational in that specific moment of threatening.


School shooting Severe targeted violence Threat assessment Warning behaviour Codebook Interrater reliability Scientific instrument 



The present study is part of the interdisciplinary 3 year research project TARGET (2013–2016), funded by the Federal Ministry of Education and Research (BMBF) of Germany (funding code 13N12646).


  1. Ahlig, N., Leuschner, V., & Scheithauer, H. (2016). ‘Entwicklung eines Instruments zur Erfassung des Subjektiven Sicherheitsgefühls von Lehrkräften im Zusammenhang mit School Shootings und schwerer, zielgerichteter Schulgewalt’ (SG-L-SS). Konstruktion und Reliabilität einer delikt spezifischen Messung. Forensische Psychiatrie, Psychologie, Kriminologie, 1–14.Google Scholar
  2. Ahlig, N., Fiedler, N., Meloy, R., Hoffmann, J., Leuschner, V., & Scheithauer, H. (under review). Approaches to distinguish between serious and non-serious school shooting threats: a systematic review of empirical studies.Google Scholar
  3. Bondü, R., & Scheithauer, H. (2014). Leaking and death-threats by students: A study in German schools. School Psychology International, 35(6), 592–608. Scholar
  4. Borum, R. (2000). Assessing violence risk among youth. Journal of Clinical Psychology, 56, 1263–1288.CrossRefGoogle Scholar
  5. De Becker, G. (1999). Mut zur Angst. Hamburg: Krüger.Google Scholar
  6. Cornell, D. G. (2003). Guidelines for responding to student threats of violence. Journal of Educational Administration, 41(6), 705–719.CrossRefGoogle Scholar
  7. Cornell, D., & Allen, K. (2011). Development, evaluation, and future directions of the Virginia student threat assessment guidelines. Journal of School Violence, 10(1), 88–106.CrossRefGoogle Scholar
  8. Cornell, D. G., Sheras, P. L., Kaplan, S., McConville, D., Douglass, J., Elkon, A., & Cole, J. (2004). Guidelines for student threat assessment: Field-test findings. School Psychology Review, 33(4), 527–546.Google Scholar
  9. Cornell, D., & Sheras, P. (2006). Guidelines for responding to student threats of violence. Longmont, CO: Sopris West.Google Scholar
  10. Cornell, D., Sheras, P., Gregory, A., & Fan, X. (2009). A retrospective study of school safety conditions in high schools using the Virginia threat assessment guidelines versus alternative approaches. School Psychology Quarterly, 24(2), 119.CrossRefGoogle Scholar
  11. Göbel, K., Sommer, F., Taefi, A., Stetten, L., Ahlig, N., Allwinn, M., Leuschner, V., & Scheithauer, H. (2016). Entwicklung und Reliabilitätsprüfung eines interdisziplinären Codebooks zur wissenschaftlichen Analyse von Strafakten zu Mord- und Totschlag Delikten. Rechtspsychologie, 4, 429–447.CrossRefGoogle Scholar
  12. Gwet, K. L. (2010). Inter-rater reliability publications. Inter-rater reliability with R - R functions for calculating agreement coefficients. Advanced Analytics, LLC: Gaithersburg. Retrieved from Scholar
  13. Hoffmann, J., Roshdi, K., and Robertz, F. (2009). Zielgerichtete schwere Gewalt und Amok an Schulen: Eine empirische Studie zur Prävention schwerer Gewalttaten. [Severe targeted violence and school shootings: An empirical study to prevent homicidal 24 violence]. Kriminalistik, 4, 196–204.Google Scholar
  14. Kaplan, S. G., & Cornell, D. G. (2005). Threats of violence by students in special education. Behavioral Disorder, 31(1), 107–119.CrossRefGoogle Scholar
  15. Kurasaki, K. S. (2000). Intercoder reliability for validating conclusions drawn from open-ended interview data. Field Methods, 12(3), 179–194.CrossRefGoogle Scholar
  16. Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.CrossRefGoogle Scholar
  17. Leuschner, V., Bondü, R., Schroer-Hippel, M., Panno, J., Neumetzler, K., Fisch, S., & Scheithauer, H. (2011). Prevention of homicidal violence in schools in Germany: The Berlin leaking project and the networks against school shootings project (NETWASS). New Directions for Youth Development, 2011(129), 61–78.CrossRefGoogle Scholar
  18. Leuschner, V., Bondü, R., Allroggen, M., and Scheithauer, H. (2016). Leaking: Häufigkeit und Korrelate von Ankündigungen und Androhungen tödlicher Gewalt nach Meldungen Berliner Schulen zwischen 1996 und 2007. [Leaking: Frequency and correlates of announcements and threats of homicidal violence reported by Berlin schools between 1996 and 2007]. Zeitschrift für Kinder- und Jugendpsychiatrie und Psychotherapie, 44, 208–219.CrossRefGoogle Scholar
  19. Leuschner, V., Fiedler, N., Schultze, M., Ahlig, N., Göbel, K., Sommer, F., Scholl, J., Cornell, D., & Scheithauer, H. (2017). Prevention of targeted school violence by responding to students’ psychosocial crises: The NETWASS program. Child Development, 88(1), 68–82.CrossRefGoogle Scholar
  20. Madfis, E. (2016). In search of meaning: Are school rampage shootings random and senseless violence? The Journal of Psychology, 1–15.Google Scholar
  21. Meloy, R., Hoffmann, J., Guldimann, A., & James, D. (2012). The role of warning behaviors in threat assessment: An exploration and suggested typology. Behavioral Sciences & the Law, 30(3), 256–279.CrossRefGoogle Scholar
  22. Meloy, J. R., Hoffmann, J., Roshdi, K., & Guldimann, A. (2014). Some warning behaviors discriminate between school shooters and other students of concern. Journal of Threat Assessment and Management, 1(3), 203–211.CrossRefGoogle Scholar
  23. Meloy, J. R., & O’Toole, M. E. (2011). The concept of leakage in threat assessment. Behavioral Sciences & the Law, 29(4), 513–527.CrossRefGoogle Scholar
  24. Muschert, G.W. (2013). School shootings as mediatized violence. In School shootings (pp. 265–281). New York: Springer.Google Scholar
  25. Nekvasil, E. K., & Cornell, D. G. (2012). Student reports of peer threats of violence: Prevalence and outcomes. Journal of School Violence, 11(4), 357–375.CrossRefGoogle Scholar
  26. Newman, K., & Fox, C. (2009). Repeat tragedy: Rampage shootings in American high school and college settings, 2002-2008. American Behavioral Scientist, 52, 1286–1308.CrossRefGoogle Scholar
  27. Oksanen, A., Kaltiala-Heino, R., Holkeri, E., & Lindberg, N. (2015). School shooting threats as a national phenomenon: Comparison of police reports and psychiatric reports in Finland. Journal of Scandinavian Studies in Criminology and Crime Prevention, 16(2), 145–159.CrossRefGoogle Scholar
  28. O’Toole, M. E. (1999). The school shooter: A threat assessment perspective. Quantico, VA: National Center for the Analysis of Violent Crime, Federal Bureau of Investigation.Google Scholar
  29. Pollack, W. S., Modzeleski, W., & Rooney, G. (2008). Prior knowledge of potential school based violence: Information students learn may prevent a targeted attack. Washington, DC: U.S. Department of Education.Google Scholar
  30. Reddy, M., Borum, R., Berglund, J., Vossekuil, B., Fein, R., & Modzeleski, W. (2001). Evaluating risk for targeted violence in schools: Comparing risk assessment, threat assessment, and other approaches. Psychology in the Schools, 38(2), 157–172.CrossRefGoogle Scholar
  31. Ryan-Arredondo, K., Renouf, K., Egyed, C., Doxey, M., Dobbins, M., Sanchez, S., & Rakowitz, B. (2001). Threats of violence in schools: The Dallas independent School District’s response. Psychology in the Schools, 38(2), 185–196.CrossRefGoogle Scholar
  32. Scheithauer, H. & Bondü, R. (2011). Amoklauf und School Shooting. Bedeutung, Hintergründe und Prävention. Göttingen: Vandenhoeck & Ruprecht.Google Scholar
  33. Sommer, F., Fiedler, N., Leuschner, V., & Scheithauer, H. (2016). Strukturen zur Identifikation und Bewertung krisenhafter Entwicklungen im Kindes- und Jugendalter - Das NETWASS-Krisenprä-ventionsverfahren für Schulen. Zeitschrift für Kinder- und Jugendpsychiatrie und Psychotherapie, 44, 198–207.CrossRefGoogle Scholar
  34. Strong, K., & Cornell, D. (2008). Student threat assessment in Memphis City schools: A descriptive report. Behavioral Disorders, 34(1), 42–54.CrossRefGoogle Scholar
  35. TARGET Research Group. (2015). TARGET-Codebook zur Aktenanalyse. Berlin: Freie Universität Berlin.Google Scholar
  36. Twemlow, S. W., Fonagy, P., Sacco, F. C., O’Toole, M. E., Vernberg, E., & Jellinek, M. S. (2002). Premeditated mass shootings in schools: Threat assessment. Journal of the American Academy of Child & Adolescent Psychiatry, 41(4), 475–477.CrossRefGoogle Scholar
  37. Webster, C., Douglas, K., Eaves, D., & Hart, S. (1997). HCR-20 assessing risk for violence, version 2. Burnaby, BC: Mental Health, Law, and Policy Institute, Simon Fraser University.Google Scholar
  38. Vossekuil, B., Fein, R. A., Reddy, M., Borum, R., & Modzeleski, W. (2002). The final report and findings of the safe school initiative: Implications for the prevention of school attacks in the United States. Washington, DC: U.S. Secret Service and U.S. Department of Education.Google Scholar
  39. Wongpakaran, N., Wongpakaran, T., Wedding, D., & Gwet, K. L. (2013). A comparison of Cohen’s kappa and Gwet’s AC1 when calculating inter-rater reliability coefficients: A study conducted with personality disorder samples. BMC Medical Research Methodology, 13(1), 61.CrossRefGoogle Scholar
  40. Wright, J. A., & Dusek, J. B. (1998). Compiling school base rates for disruptive behaviors from student disciplinary referral data. School Psychology Review, 27, 138–147.Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Nadine Ahlig
    • 1
    Email author
  • Kristin Göbel
    • 1
  • Mirko Allwinn
    • 2
  • Nora Fiedler
    • 1
  • Vincenz Leuschner
    • 3
  • Herbert Scheithauer
    • 1
  1. 1.Department of Education and Psychology, Developmental Science and Applied Developmental PsychologyFreie Universität BerlinBerlinGermany
  2. 2.Institut Psychologie und BedrohungsmanagementDarmstadtGermany
  3. 3.Department of Police and Security ManagementBerlin School of Economics and LawBerlinGermany

Personalised recommendations