Prevention Science

, Volume 20, Issue 6, pp 904–913 | Cite as

Using Smartphone Survey Data and Machine Learning to Identify Situational and Contextual Risk Factors for HIV Risk Behavior Among Men Who Have Sex with Men Who Are Not on PrEP

  • Tyler B. WrayEmail author
  • Xi Luo
  • Jun Ke
  • Ashley E. Pérez
  • Daniel J. Carr
  • Peter M. Monti


“Just-in-time” interventions (JITs) delivered via smartphones have considerable potential for reducing HIV risk behavior by providing pivotal support at key times prior to sex. However, these programs depend on a thorough understanding of when risk behavior is likely to occur to inform the timing of JITs. It is also critical to understand the most important momentary risk factors that may precede HIV risk behavior, so that interventions can be designed to address them. Applying machine learning (ML) methods to ecological momentary assessment data on HIV risk behaviors can help answer both questions. Eighty HIV-negative men who have sex with men (MSM) who were not on PrEP completed a daily diary survey each morning and an experience sampling survey up to six times per day via a smartphone application for 30 days. Random forest models achieved the highest area under the curve (AUC) values for classifying high-risk condomless anal sex (CAS). These models achieved 80% specificity at a sensitivity value of 74%. Unsurprisingly, the most important contextual risk factors that aided in classification were participants’ plans and intentions for sex, sexual arousal, and positive affective states. Findings suggest that survey data collected throughout the day can be used to correctly classify about three of every four high-risk CAS events, while incorrectly classifying one of every five non-CAS days as involving high-risk CAS. A unique set of risk factors also often emerge prior to high-risk CAS events that may be useful targets for JITs.


HIV Sexual behavior Men who have sex with men Machine learning Ecological momentary assessment 



This manuscript was supported by P01AA019072 (to PM) and L30AA023336 (to TW) from the National Institute on Alcohol Abuse and Alcoholism.

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.

Research Involving Human Participants and/or Animals

All procedures in this study were approved by the Brown University Institutional Review Board.

Informed Consent

All participants in this study provided informed consent prior to enrollment.

Supplementary material

11121_2019_1019_MOESM1_ESM.docx (18 kb)
ESM 1 (DOCX 17 kb)


  1. Ajzen, I. (1991). The theory of planned behavior. Organizational behavior and human decision processes, 50, 179–211.CrossRefGoogle Scholar
  2. Aspinwall, L. G. (1998). Rethinking the role of positive affect in self-regulation. Motivation and Emotion, 22, 1–32.CrossRefGoogle Scholar
  3. Bae, S., Ferreira, D., Suffoletto, B., Puyana, J. C., Kurtz, R., Chung, T., & Dey, A. K. (2017). Detecting drinking episodes in young adults using smartphone-based sensors. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 1, 5.Google Scholar
  4. Bi, J., Sun, J., Wu, Y., Tennen, H., & Armeli, S. (2013). A machine learning approach to college drinking prediction and risk factor identification. ACM Transactions on Intelligent Systems and Technology (TIST), 4, 72.Google Scholar
  5. Breiman, L. (2001). Statistical modeling: the two cultures (with comments and a rejoinder by the author). Statistical Science, 16, 199–231.CrossRefGoogle Scholar
  6. Centers for Disease Control and Prevention. (2013). HIV prevention: progress to date. Atlanta: U.S. Department of Health and Human Services.Google Scholar
  7. Centers for Disease Control and Prevention. (2016a). Lifetime risk of HIV diagnosis. Retrieved from:
  8. Centers for Disease Control and Prevention. (2016b). Trends in HIV diagnoses, 2005–2014. Retrieved from:
  9. Centers for Disease Control and Prevention. (2016c). As many as 185,000 new HIV infections in the U.S. could be prevented by expanding testing, treatment, PrEP. Retrieved from:
  10. Centers for Disease Control and Prevention. (2017). HIV among African American gay and bisexual men. Atlanta: National Center for HIV/AIDS, Viral Hepatitis, STD, and TB Prevention.Google Scholar
  11. Choe, E. K., Lee, N. B., Lee, B., Pratt, W., & Kientz, J. A. (2014). Understanding quantified-selfers’ practices in collecting and exploring personal data. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1143–1152). New York: ACM.Google Scholar
  12. Dey, A. K., Wac, K., Ferreira, D., Tassini, K., Hong, J. H., & Ramos, J. (2011). Getting closer: an empirical investigation of the proximity of user to their smart phones. Proceedings of the 13th International Conference on Ubiquitous Computing (pp. 163–172). New York: ACM.Google Scholar
  13. Etchings, J. A. (2017). Strategies in biomedical data science: driving force for innovation. Hoboken: John Wiley & Sons.Google Scholar
  14. George, W. H., Davis, K. C., Norris, J., Heiman, J. R., Stoner, S. A., Schacht, R. L., et al. (2009). Indirect effects of acute alcohol intoxication on sexual risk-taking: the roles of subjective and physiological sexual arousal. Archives of Sexual Behavior, 38, 498–513.CrossRefPubMedGoogle Scholar
  15. Goldstein, S. P., Evans, B. C., Flack, D., Juarascio, A., Manasse, S., Zhang, F., & Forman, E. M. (2017). Return of the JITAI: applying a just-in-time adaptive intervention framework to the development of m-health solutions for addictive behaviors. International Journal of Behavioral Medicine, 24, 673–682.CrossRefPubMedPubMedCentralGoogle Scholar
  16. Google Developers. (2018). Google APIs for Android: detected activity. Retrieved from:
  17. Grov, C., Golub, S. A., Mustanski, B., & Parsons, J. T. (2010). Sexual compulsivity, state affect, and sexual risk behavior in a daily diary study of gay and bisexual men. Psychology of Addictive Behaviors, 24, 487–497.CrossRefPubMedGoogle Scholar
  18. Grov, C., Rendina, H. J., Ventuneac, A., & Parsons, J. T. (2016). Sexual behavior varies between same-race and different-race partnerships: a daily diary study of highly sexually active Black, Latino, and White gay and bisexual men. Archives of Sexual Behavior, 45, 1453–1462.CrossRefPubMedGoogle Scholar
  19. Heron, K. E., & Smyth, J. M. (2010). Ecological momentary interventions: incorporating mobile technology into psychosocial and health behaviour treatments. British Journal of Health Psychology, 15, 1–39.CrossRefGoogle Scholar
  20. Hjorthøj, C. R., Hjorthøj, A. R., & Nordentoft, M. (2012). Validity of timeline follow-back for self-reported use of cannabis and other illicit substances—systematic review and meta-analysis. Addictive Behaviors, 37, 225–233.CrossRefPubMedGoogle Scholar
  21. Hosek, S., Landovitz, R., Rudy, B., Kapogiannis, B., Siberry, G., & Rutledge, B. (2016). An HIV pre-exposure prophylaxis (PrEP) demonstration project and safety study for adolescent MSM ages 15–17 in the United States (ATN 113). Paper presented at the International AIDS Conference.Google Scholar
  22. Intille, S., Haynes, C., Maniar, D., Ponnada, A., & Manjourides, J. (2016). μEMA: microinteraction-based ecological momentary assessment (EMA) using a smartwatch. Paper presented at the proceedings of the 2016 ACM international joint conference on pervasive and ubiquitous computing.Google Scholar
  23. LiKamWa, R., Liu, Y., Lane, N. D., & Zhong, L. (2013). Moodscope: building a mood sensor from smartphone usage patterns. Paper presented at the proceeding of the 11th annual international conference on Mobile systems, applications, and services.Google Scholar
  24. Liu, A., Glidden, D. V., Anderson, P. L., Amico, K. R., McMahan, V., Mehrotra, M., et al. (2014). Patterns and correlates of PrEP drug detection among MSM and transgender women in the Global iPrEx Study. Journal of Acquired Immune Deficiency Syndromes (1999), 67, 528–537.CrossRefGoogle Scholar
  25. Mayer, K., Maloney, K., Levine, K., King, D., Grasso, C., Krakower, D., & Boswell, S. (2016). HIV infection and PrEP use are independently associated with diagnoses of bacterial sexually transmitted infections in men accessing care at a Boston community health center. Paper presented at the ID week, New Orleans, LA.Google Scholar
  26. Muaremi, A., Arnrich, B., & Tröster, G. (2013). Towards measuring stress with smartphones and wearable devices during workday and sleep. BioNanoScience, 3, 172–183.CrossRefPubMedPubMedCentralGoogle Scholar
  27. Mustanski, B. (2007). The influence of state and trait affect on HIV risk behaviors: a daily diary study of MSM. Health Psychology, 26, 618–626.CrossRefPubMedGoogle Scholar
  28. Nahum-Shani, I., Smith, S. N., Spring, B. J., Collins, L. M., Witkiewitz, K., Tewari, A., & Murphy, S. A. (2017). Just-in-time adaptive interventions (JITAIs) in mobile health: key components and design principles for ongoing health behavior support. Annals of Behavioral Medicine, 52, 446–462.Google Scholar
  29. Newcomb, M. E., & Mustanski, B. (2013). Diaries for observation or intervention of health behaviors: factors that predict reactivity in a sexual diary study of men who have sex with men. Annals of Behavioral Medicine, 47, 325–334.Google Scholar
  30. Onnela, J. P., & Rauch, S. L. (2016). Harnessing smartphone-based digital phenotyping to enhance behavioral and mental health. Neuropsychopharmacology, 41, 1691–1696.CrossRefPubMedPubMedCentralGoogle Scholar
  31. Paolillo, E. W., Obermeit, L. C., Tang, B., Depp, C. A., Vaida, F., Moore, D. J., & Moore, R. C. (2018). Smartphone-based ecological momentary assessment (EMA) of alcohol and cannabis use in older adults with and without HIV infection. Addictive Behaviors, 83, 102–108.Google Scholar
  32. Parsons, J. T., Rendina, H. J., Grov, C., Ventuneac, A., & Mustanski, B. (2015). Accuracy of highly sexually active gay and bisexual men's predictions of their daily likelihood of anal sex and its relevance for intermittent event-driven HIV Pre-Exposure Prophylaxis. Journal of Acquired Immune Deficiency Syndromes (JAIDS), 68, 449–455.Google Scholar
  33. Punyacharoensin, N., Edmunds, W. J., De Angelis, D., Delpech, V., Hart, G., Elford, J., et al. (2016). Effect of pre-exposure prophylaxis and combination HIV prevention for men who have sex with men in the UK: a mathematical modelling study. The lancet HIV, 3, e94–e104.CrossRefPubMedGoogle Scholar
  34. Puterman, E. (2009). Bringing risk prevention into the bedroom: sex motives and risky behaviors in men who have sex with men. Doctoral dissertation, University of British Columbia.Google Scholar
  35. Rosser, B. S., Wilkerson, J. M., Smolenski, D. J., Oakes, J. M., Konstan, J., Horvath, K. J., et al. (2011). The future of Internet-based HIV prevention: a report on key findings from the Men’s INTernet (MINTS-I, II) Sex Studies. AIDS and Behavior, 15, 91–100.CrossRefGoogle Scholar
  36. Sander, P. M., Cole, S. R., Stall, R. D., Jacobson, L. P., Eron, J. J., Napravnik, S., et al. (2013). Joint effects of alcohol consumption and high-risk sexual behavior on HIV seroconversion among men who have sex with men. Aids, 27, 815–823.CrossRefPubMedPubMedCentralGoogle Scholar
  37. Shiffman, S. (2009). Ecological momentary assessment (EMA) in studies of substance use. Psychological Assessment, 21, 486–497.CrossRefPubMedPubMedCentralGoogle Scholar
  38. Shiffman, S., Stone, A. A., & Hufford, M. R. (2008). Ecological momentary assessment. Annual Review of Clinical Psychology, 4, 1–32 Retrieved from:
  39. Shoaib, M., Bosch, S., Scholten, H., Havinga, P. J., & Incel, O. D. (2015). Towards detection of bad habits by fusing smartphone and smartwatch sensors. 2015 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops) (pp. 591–596). St. Louis: IEEE.Google Scholar
  40. Siegler, A., Mouhanna, F., Giler, R., McCallister, S., Yeung, H., Jones, J., … Sullivan, P. S. (2018). Distribution of active PrEP prescriptions and the PrEP-to-need ratio, US, Q2 2017. Paper presented at the conference on retroviruses and opportunistic infections (CROI), Boston, MA.Google Scholar
  41. Simons, J. S., Wills, T. A., Emery, N. N., & Marks, R. M. (2015). Quantifying alcohol consumption: self-report, transdermal assessment, and prediction of dependence symptoms. Addictive Behaviors, 50, 205–212.CrossRefPubMedPubMedCentralGoogle Scholar
  42. Smyth, J. M., & Heron, K. E. (2016). Is providing mobile interventions" just-in-time" helpful? An experimental proof of concept study of just-in-time intervention for stress management. Paper presented at IEEE Wireless Health 2016 (pp. 1–7). Retrieved from:
  43. Sobell, L. C., & Sobell, M. B. (1992). Timeline follow-back measuring alcohol consumption (pp. 41-72): Springer.Google Scholar
  44. Spruijt-Metz, D., & Nilsen, W. (2014). Dynamic models of behavior for just-in-time adaptive interventions. IEEE Pervasive Computing, 13, 13–17.CrossRefGoogle Scholar
  45. Spruijt-Metz, D., Hekler, E., Saranummi, N., Intille, S., Korhonen, I., Nilsen, W., et al. (2015). Building new computational models to support health behavior change and maintenance: new opportunities in behavioral research. Translational Behavioral Medicine, 5, 335–346.CrossRefPubMedPubMedCentralGoogle Scholar
  46. Stone, A. A., & Shiffman, S. (1994). Ecological momentary assessment (EMA) in behavorial medicine. Annals of Behavioral Medicine, 16, 199–202.Google Scholar
  47. Swendeman, D., Ramanathan, N., Baetscher, L., Medich, M., Scheffler, A., Comulada, W. S., & Estrin, D. (2015). Smartphone self-monitoring to support self-management among people living with HIV: perceived benefits and theory of change from a mixed-methods, randomized pilot study. Journal of Acquired Immune Deficiency Syndromes (JAIDS), 69, S80–S91.Google Scholar
  48. US Public Health Service. (2018). Preexposure prophylaxis for the prevention of HIV infection in the United States - 2017 update. New York: US Department of Health and Human Services.Google Scholar
  49. Vosburgh, H. W., Mansergh, G., Sullivan, P. S., & Purcell, D. W. (2012). A review of the literature on event-level substance use and sexual risk behavior among men who have sex with men. AIDS and Behavior, 16, 1394–1410.CrossRefPubMedGoogle Scholar
  50. Wenze, S. J., & Miller, I. W. (2010). Use of ecological momentary assessment in mood disorders research. Clinical Psychology Review, 30, 794–804.CrossRefGoogle Scholar
  51. Wray, T. B., Merrill, J., & Monti, P. M. (2015). Using ecological momentary assessment (EMA) to assess situation-level risk factors for heavy drinking and alcohol-related consequences. Alcohol Research & Health, 36, 19–27.Google Scholar
  52. Wray, T. B., Kahler, C. W., & Monti, P. M. (2016). Using ecological momentary assessment (EMA) to study sex events among very high-risk men who have sex with men (MSM). AIDS and Behavior, 20, 2231–2242. Scholar
  53. Wray, T. B., Adia, A. C., Pérez, A. E., Simpanen, E. M., Woods, L.-A., Celio, M. A., & Monti, P. M. (2018). Timeline: A web application for assessing the timing and details of health behaviors. The American Journal of Drug and Alcohol Abuse, 45, 141–150.
  54. Wray, T. B., Celio, M. A., Pérez, A. E., DiGuiseppi, G. T., Carr, D. J., Woods, L. A., & Monti, P. M. (2019). Causal effects of alcohol intoxication on sexual risk intentions and condom negotiation skills among high-risk men who have sex with men (MSM). AIDS and Behavior, 23, 161–174.Google Scholar
  55. Yang, C., Linas, B., Kirk, G., Bollinger, R., Chang, L., Chander, G., … & Latkin, C. (2015). Feasibility and acceptability of smartphone-based ecological momentary assessment of alcohol use among African American men who have sex with men in Baltimore. JMIR mHealth and uHealth, 3, e67.Google Scholar

Copyright information

© Society for Prevention Research 2019

Authors and Affiliations

  1. 1.Department of Behavioral and Social Sciences, Center for Alcohol and Addictions StudiesBrown University School of Public HealthProvidenceUSA
  2. 2.Department of BiostatisticsBrown University School of Public HealthProvidenceUSA
  3. 3.Department of Biostatistics and Data Science, School of Public HealthThe University of Texas Health Science Center at HoustonHoustonUSA
  4. 4.Department of Social and Behavioral SciencesUniversity of California, San FranciscoSan FranciscoUSA

Personalised recommendations