Cognitive Therapy and Research

, Volume 42, Issue 6, pp 747–757 | Cite as

Personalizing Affective Stimuli Using a Recommender Algorithm: An Example with Threatening Words for Trauma Exposed Populations

  • Andrea N. NilesEmail author
  • Aoife O’Donovan
Original Article


Experimental paradigms used in affective and clinical science often use stimuli such as images, scenarios, videos, or words to elicit emotional responses in study participants. Choosing appropriate stimuli that are highly evocative is essential to the study of emotional processes in both healthy and clinical populations. Selecting one set of stimuli that will be relevant for all subjects can be challenging because not every person responds the same way to a given stimulus. Machine learning can facilitate the personalization of such stimuli. The current study applied a novel statistical approach called a recommender algorithm to the selection of highly threatening words for a trauma-exposed population (N = 837). Participants rated 513 threatening words, and we trained a user–user collaborative filtering recommender algorithm. The algorithm uses similarities between individuals to predict ratings for unrated words. We compared threat ratings for algorithm-based word selection to a random word set, a word set previously used in research, and trauma-specific word sets. Algorithm-selected personalized words were more threatening compared to non-personalized words with large effects (ds = 2.10–2.92). Recommender algorithms can automate the personalization of stimuli from a large pool of possible stimuli to maximize emotional reactivity in research paradigms. These methods also hold potential for the personalization of behavioral treatments administered remotely where a provider is not available to tailor an intervention to the individual. The word personalization algorithm is available for use online (


Anxiety Posttraumatic stress disorder Emotion Affect Recommender algorithm Trauma Personalization Personalized medicine 



The current project was funded by the UCSF Digital Mental Health Resource Allocation Program. Dr. Niles is funded by the San Francisco VA Advanced Women’s Health Fellowship and Dr. O'Donovan is funded by a K01 Award from the National Institute of Mental Health (K01MH109871) and a University of California Hellman Award. We also acknowledge Vivien Li for her significant contributions during the data collection phase of this project.


Funding was provided by University of California, San Francisco (Grant No. 7710 133105 7500634 45), the San Francisco VA Advanced Women's Health Fellowship, a K01 Career Development Award (K01MH109871) and a University of California Hellman Award.

Compliance with Ethical Standards

Conflict of Interest

Drs. Niles and O’Donovan declare that they have no conflict of interest.

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Animal Rights Statements

No animal studies were carried out by the authors for this article.


  1. Aggarwal, C. (2016). Recommender systems: The textbook. Yorktown Heights, NY: Springer.CrossRefGoogle Scholar
  2. Amir, N., Beard, C., Burns, M., & Bomyea, J. (2009). Attention modification program in individuals with generalized anxiety disorder. Journal of Abnormal Psychology, 118(1), 28–33. Scholar
  3. Amir, N., Kuckertz, J. M., & Strege, M. V. (2016). A pilot study of an adaptive, idiographic, and multi-component attention bias modification program for social anxiety disorder. Cognitive Therapy and Research, 40(5), 661–671. Scholar
  4. Amir, N., & Taylor, C. T. (2012). Combining computerized home-based treatments for generalized anxiety disorder: An attention modification program and cognitive behavioral therapy. Behavior Therapy, 43(3), 546–559. Scholar
  5. Beck, J. G., Freeman, J. B., Shipherd, J. C., Hamblen, J. L., & Lackner, J. M. (2001). Specificity of Stroop interference in patients with pain and PTSD. Journal of Abnormal Psychology, 110(4), 536–543.CrossRefGoogle Scholar
  6. Blix, I., & Brennen, T. (2011). Intentional forgetting of emotional words after trauma: A study with victims of sexual assault. Frontiers in Psychology. Scholar
  7. Bradley, M., & Lang, P. (1994). Measuring emotion: The self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry, 25(1), 49–59.CrossRefGoogle Scholar
  8. Bradley, M., & Lang, P. (1999). Affective norms for English words (ANEW): Instruction manual and affective ratings. Gainesville: University of FloridaGoogle Scholar
  9. Buhrmester, M., Kwang, T., & Gosling, S. D. (2011). Amazon’s Mechanical Turk. Perspectives on Psychological Science, 6(1), 3–5. Scholar
  10. Bunnell, S. (2007). The impact of abuse exposure on memory processes and attentional biases in a college-aged sample. Ann Arbor Dissertations & Theses.Google Scholar
  11. Carlson, E. B., Smith, S. R., Palmieri, P. A., Dalenberg, C., Ruzek, J. I., Kimerling, R., … Spain, D. A. (2011). Development and validation of a brief self-report measure of trauma exposure: The trauma history screen. Psychological Assessment, 23(2), 463–477. Scholar
  12. Carroll, J. M., Yik, M. S. M., Russell, J. A., & Barrett, L. F. (1999). On the psychometric principles of affect. Review of General Psychology, 3(1), 14–22.CrossRefGoogle Scholar
  13. Chambers, R., Lo, B. C. Y., & Allen, N. B. (2008). The impact of intensive mindfulness training on attentional control, cognitive style, and affect. Cognitive Therapy and Research, 32(3), 303–322. Scholar
  14. Colin, M., Lih, Y. S., Elizabeth, M. R., & Lynlee, W. C. (2007). Internet-delivered assessment and manipulation of anxiety-linked attentional bias: Validation of a free-access attentional probe software package. Behavior Research Methods, 39(3), 533–538.CrossRefGoogle Scholar
  15. Ehring, T., & Ehlers, A. (2011). Enhanced priming for trauma-related words predicts posttraumatic stress disorder. Journal of Abnormal Psychology, 120(1), 234–239. Scholar
  16. Fleischer, A., Mead, A. D., & Huang, J. (2015). Inattentive responding in MTurk and other online samples. Industrial and Organizational Psychology, 8(2), 196–202. Scholar
  17. Fridrici, C., Leichsenring-Driessen, C., Driessen, M., Wingenfeld, K., Kremer, G., & Beblo, T. (2013). The individualized alcohol Stroop task: No attentional bias toward personalized stimuli in alcohol-dependents. Psychology of Addictive Behaviors: Journal of the Society of Psychologists in Addictive Behaviors, 27(1), 62–70. Scholar
  18. Hazen, R. A., Vasey, M. W., & Schmidt, N. B. (2009). Attentional retraining: A randomized clinical trial for pathological worry. Journal of Psychiatric Research, 43(6), 627–633. Scholar
  19. Kensinger, E. A., & Corkin, S. (2003). Memory enhancement for emotional words: Are emotional words more vividly remembered than neutral words? Memory & Cognition, 31(8), 1169–1180. Scholar
  20. Kilpatrick, D. G., Resnick, H. S., Milanak, M. E., Miller, M. W., Keyes, K. M., & Friedman, M. J. (2013). National estimates of exposure to traumatic events and PTSD prevalence using DSM-IV and DSM-5 criteria. Journal of Traumatic Stress, 26(5), 537–547. Scholar
  21. Koren, Y., Bell, R., & Volinsky, C. (2009). Matrix factorization techniques for recommender systems. Computer, 42(8), 30–37. Scholar
  22. Kousta, S.-T., Vinson, D. P., & Vigliocco, G. (2009). Emotion words, regardless of polarity, have a processing advantage over neutral words. Cognition, 112(3), 473–481. Scholar
  23. Lang, P. J., Bradley, M. M., & Cuthbert, B. N. (1999). International affective picture system (IAPS): Technical manual and affective ratings. Gainesville, FL: The Center for Research in Psychophysiology, University of Florida.Google Scholar
  24. MacLeod, C., Rutherford, E., Campbell, L., Ebsworthy, G., & Holker, L. (2002). Selective attention and emotional vulnerability: Assessing the causal basis of their association through the experimental manipulation of attentional bias. The Journal of Abnormal Psychology, 111(1), 107–123. Scholar
  25. McGeary, J. E., Meadows, S. P., Amir, N., & Gibb, B. E. (2014). Computer-delivered, home-based, attentional retraining reduces drinking behavior in heavy drinkers. Psychology of Addictive Behaviors, 28(2), 559–562. Scholar
  26. Nadimi-Shahraki, M.-H., & Bahadorpour, M. (2014). Cold-start problem in collaborative recommender systems: Efficient methods based on ask-to-rate technique. Journal of Computing and Information Technology, 22(2), 105–113. Scholar
  27. R Core Team. (2013). R: A language and environment for statistical computing. Vienna: R Development Core TeamGoogle Scholar
  28. Rashid, A. M., Albert, I., Cosley, D., Lam, S. K., McNee, S. M., Konstan, J. A., & Riedl, J. (2002). Getting to know you: Learning new user preferences in recommender systems. In Proceedings of the 7th international conference on intelligent user interfaces (pp. 127–134). New York, NY, USA: ACM.
  29. Roberts, K. E., Hart, T. A., & Eastwood, J. D. (2010). Attentional biases to social and health threat words in individuals with and without high social anxiety or depression. Cognitive Therapy and Research, 34(4), 388–399. Scholar
  30. Schoorl, M., Putman, P., Mooren, T. M., Van Der Werff, S., & Van Der Does, W. (2014). Attentional bias modification in Dutch veterans with posttraumatic stress disorder: A case series with a personalized treatment version. Journal of Traumatic Stress, 27(2), 240–243. Scholar
  31. Shao, Z., Janse, E., Visser, K., & Meyer, A. S. (2014). What do verbal fluency tasks measure? Predictors of verbal fluency performance in older adults. Frontiers in Psychology. Scholar
  32. Siegle, G. J., Steinhauer, S. R., Thase, M. E., Stenger, V. A., & Carter, C. S. (2002). Can’t shake that feeling: event-related fMRI assessment of sustained amygdala activity in response to emotional information in depressed individuals. Biological Psychiatry, 51(9), 693–707. Scholar
  33. StataCorp. (2015). Stata statistical software: Release 14. College Station, TX: StataCorp LP.Google Scholar
  34. Swick, D. (2009). The separate and cumulative effects of TBI and PTSD on cognitive function and emotional control (Annual Report No. W81XWH-08-2-0086). Martinez, CA: Veterans Health Administration.Google Scholar

Copyright information

© This is a U.S. Government work and not under copyright protection in the US; foreign copyright protection may apply  2018

Authors and Affiliations

  1. 1.Department of Psychiatry and Weill Institute for NeurosciencesUniversity of California, San FranciscoSan FranciscoUSA
  2. 2.San Francisco Veterans Affairs Medical CenterSan FranciscoUSA

Personalised recommendations