Machine learning models reveal neurocognitive impairment type and prevalence are associated with distinct variables in HIV/AIDS

  • Wei Tu
  • Patricia A. Chen
  • Noshin Koenig
  • Daniela Gomez
  • Esther Fujiwara
  • M. John Gill
  • Linglong KongEmail author
  • Christopher PowerEmail author


Neurocognitive impairment (NCI) among HIV-infected patients is heterogeneous in its reported presentations and frequencies. To determine the prevalence of NCI and its associated subtypes as well as predictive variables, we investigated patients with HIV/AIDS receiving universal health care. Recruited adult HIV-infected subjects underwent a neuropsychological (NP) test battery with established normative (sex-, age-, and education-matched) values together with assessment of their demographic and clinical variables. Three patient groups were identified including neurocognitively normal (NN, n = 246), HIV-associated neurocognitive disorders (HAND, n = 78), and neurocognitively impaired-other disorders (NCI-OD, n = 46). Univariate, multiple logistic regression and machine learning analyses were applied. Univariate analyses showed variables differed significantly between groups including birth continent, quality of life, substance use, and PHQ-9. Multiple logistic regression models revealed groups again differed significantly for substance use, PHQ-9 score, VACS index, and head injury. Random forest (RF) models disclosed that classification algorithms distinguished HAND from NN and NCI-OD from NN with area under the curve (AUC) values of 0.87 and 0.77, respectively. Relative importance plots derived from the RF model exhibited distinct variable rankings that were predictive of NCI status for both NN versus HAND and NN versus NCI-OD comparisons. Thus, NCI was frequently detected (33.5%) although HAND prevalence (21%) was lower than in several earlier reports underscoring the potential contribution of other factors to NCI. Machine learning models uncovered variables related to individual NCI types that were not identified by univariate or multiple logistic regression analyses, highlighting the value of other approaches to understanding NCI in HIV/AIDS.


Neurocognitive impairment HIV-associated neurocognitive disorders Machine learning Neuropsychology Comorbidity 



A Canadian Institutes of Health Research, Emerging Team Grant (EF, MJG, and CP) supported these studies. The authors thank the patients and staff at the Southern Alberta Clinic for their willingness to participate in the study and Brenda Beckthold for the technical assistance.

Compliance with ethical standards

The University of Calgary Ethics Committee (REB #-130615) approved the study and written consent was obtained from all patients.

Conflict of interest

The authors declare that they have no conflict of interest.

Supplementary material

13365_2019_791_MOESM1_ESM.docx (26 kb)
ESM 1 (DOCX 26 kb)
13365_2019_791_MOESM2_ESM.pptx (106 kb)
ESM 2 (PPTX 105 kb)


  1. Antinori A, Arendt G, Becker JT, Brew BJ, Byrd DA, Cherner M, Clifford DB, Cinque P, Epstein LG, Goodkin K, Gisslen M, Grant I, Heaton RK, Joseph J, Marder K, Marra CM, McArthur JC, Nunn M, Price RW, Pulliam L, Robertson KR, Sacktor N, Valcour V, Wojna VE (2007) Updated research nosology for HIV-associated neurocognitive disorders. Neurology. 69(18):1789–1799CrossRefGoogle Scholar
  2. Belete T, Medfu G, Yemiyamrew E (2017) Prevalence of HIV associated neurocognitive deficit among HIV positive people in ethiopia: a cross sectional study at Ayder Referral Hospital. Ethiop J Health Sci 27(1):67–76CrossRefGoogle Scholar
  3. Boucekine M, Loundou A, Baumstarck K, Minaya-Flores P, Pelletier J, Ghattas B, Auquier P (2013) Using the random forest method to detect a response shift in the quality of life of multiple sclerosis patients: a cohort study. BMC Med Res Methodol 13:20CrossRefGoogle Scholar
  4. Brandt J, Benedict R (2001) Hopkins verbal learning test-revised (HVLT-R). Psychological Assessment Resources, LutzGoogle Scholar
  5. Carvalhal A, Gill MJ, Letendre SL, Rachlis A, Bekele T, Raboud J et al (2016) Central nervous system penetration effectiveness of antiretroviral drugs and neuropsychological impairment in the Ontario HIV Treatment Network Cohort Study. J Neuro-Oncol 22(3):349–357Google Scholar
  6. Crane HM, Van Rompaey SE, Dillingham PW, Herman E, Diehr P, Kitahata MM (2006) A single-item measure of health-related quality-of-life for HIV-infected patients in routine clinical care. AIDS Patient Care STDs 20(3):161–174CrossRefGoogle Scholar
  7. Davis LE, Hjelle BL, Miller VE, Palmer DL, Llewellyn AL, Merlin TL, Young SA, Mills RG, Wachsman W, Wiley CA (1992) Early viral brain invasion in iatrogenic human immunodeficiency virus infection. Neurology. 42(9):1736–1739CrossRefGoogle Scholar
  8. Delis DC, Kaplan E, Kramer JH, D-KEFSSA (2001) TX: the psychological corporation. Delis-Kaplan executive function system. The Psychological Corporation, San AntonioGoogle Scholar
  9. Gisslen M, Price RW, Nilsson S (2011) The definition of HIV-associated neurocognitive disorders: are we overestimating the real prevalence? BMC Infect Dis 11:356CrossRefGoogle Scholar
  10. Gomez D, Power C, Gill MJ, Fujiwara E (2017) Determinants of risk-taking in HIV-associated neurocognitive disorders. Neuropsychology. 31(7):798–810CrossRefGoogle Scholar
  11. Gray KR, Aljabar P, Heckemann RA, Hammers A, Rueckert D (2013) Alzheimer’s disease neuroimaging I. Random forest-based similarity measures for multi-modal classification of Alzheimer’s disease. Neuroimage 65:167–175CrossRefGoogle Scholar
  12. Heaton RK, Clifford DB, Franklin DR Jr, Woods SP, Ake C, Vaida F et al (2010) HIV-associated neurocognitive disorders persist in the era of potent antiretroviral therapy: CHARTER Study. Neurology. 75(23):2087–2096CrossRefGoogle Scholar
  13. Kamminga J, Bloch M, Vincent T, Carberry A, Brew BJ, Cysique LA (2017) Determining optimal impairment rating methodology for a new HIV-associated neurocognitive disorder screening procedure. J Clin Exp Neuropsychol 39(8):753–767CrossRefGoogle Scholar
  14. Kim D, Jewison DL, Milner GR, Rourke SB, Gill MJ, Power C et al (2001) Can J Neurol Sci 28:228–231CrossRefGoogle Scholar
  15. Kongs SK, Thompson LL, Iverson GL, Heaton RK (2000) Wisconsin card sorting test-64 card version (WCST-64) Lutz, FL:PARGoogle Scholar
  16. Kroenke K, Spitzer RL, Williams JB (2001) The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 16(9):606–613CrossRefGoogle Scholar
  17. McArthur JC (1987) Neurologic manifestations of AIDS. Medicine (Baltimore) 66(6):407–437CrossRefGoogle Scholar
  18. McCombe JA, Vivithanaporn P, Gill MJ, Power C (2013) Predictors of symptomatic HIV-associated neurocognitive disorders in universal health care. HIV Med 14(2):99–107CrossRefGoogle Scholar
  19. Mothobi NZ, Brew BJ (2012) Neurocognitive dysfunction in the highly active antiretroviral therapy era. Curr Opin Infect Dis 25(1):4–9CrossRefGoogle Scholar
  20. Nakagawa F, May M, Phillips A (2013) Life expectancy living with HIV: recent estimates and future implications. Curr Opin Infect Dis 26(1):17–25CrossRefGoogle Scholar
  21. Patel A, Parikh R, Howell EH, Hsich E, Landers SH, Gorodeski EZ (2015) Mini-cog performance: novel marker of post discharge risk among patients hospitalized for heart failure. Circ Heart Fail 8(1):8–16CrossRefGoogle Scholar
  22. Pettersen JA, Jones G, Worthington C, Krentz HB, Keppler OT, Hoke A, Gill MJ, Power C (2006) Sensory neuropathy in human immunodeficiency virus/acquired immunodeficiency syndrome patients: protease inhibitor-mediated neurotoxicity. Ann Neurol 59(5):816–824CrossRefGoogle Scholar
  23. Puhan MA, Van Natta ML, Palella FJ, Addessi A, Meinert C (2010) Ocular complications of ARG. Excess mortality in patients with AIDS in the era of highly active antiretroviral therapy: temporal changes and risk factors. Clin Infect Dis 51(8):947–956CrossRefGoogle Scholar
  24. Robertson K, Liner J, Heaton R (2009) Neuropsychological assessment of HIV-infected populations in international settings. Neuropsychol Rev 19(2):232–249CrossRefGoogle Scholar
  25. Roehr B (2015) UNAIDS celebrates success in halting spread of HIV and sets goals for 2030. BMJ. 351:h3832CrossRefGoogle Scholar
  26. Rubin LH, Maki PM, Springer G, Benning L, Anastos K, Gustafson D, Villacres MC, Jiang X, Adimora AA, Waldrop-Valverde D, Vance DE, Bolivar H, Alden C, Martin EM, Valcour VG, For the Women’s Interagency HIV Study (2017) Cognitive trajectories over 4 years among HIV-infected women with optimal viral suppression. Neurology. 89(15):1594–1603CrossRefGoogle Scholar
  27. Sacktor N, Skolasky RL, Seaberg E, Munro C, Becker JT, Martin E, Ragin A, Levine A, Miller E (2016) Prevalence of HIV-associated neurocognitive disorders in the Multicenter AIDS Cohort Study. Neurology. 86(4):334–340CrossRefGoogle Scholar
  28. Saylor D, Dickens AM, Sacktor N, Haughey N, Slusher B, Pletnikov M, Mankowski JL, Brown A, Volsky DJ, McArthur JC (2016) HIV-associated neurocognitive disorder--pathogenesis and prospects for treatment. Nat Rev Neurol 12(4):234–248CrossRefGoogle Scholar
  29. Schretlen DJ, Testa SM, Pearlson GD (2010) GD. Clock-drawing test scoring approach from the Calibrated Neuropsychological Normative System. Lutz, FL: Psychological Assessment ResourcesGoogle Scholar
  30. Smail RC, Brew BJ (2018) HIV-associated neurocognitive disorder. Handb Clin Neurol 152:75–97CrossRefGoogle Scholar
  31. Smith A (1973) Symbol digit modalities test. Western Psychological Services, Los AngelesGoogle Scholar
  32. Snider WD, Simpson DM, Nielsen S, Gold JW, Metroka CE, Posner JB (1983) Neurological complications of acquired immune deficiency syndrome: analysis of 50 patients. Ann Neurol 14(4):403–418CrossRefGoogle Scholar
  33. Spudich SS, Ances BM (2017) CROI 2017: neurologic complications of HIV infection. Top Antivir Med 25(2):69–76Google Scholar
  34. Su T, Schouten J, Geurtsen GJ, Wit FW, Stolte IG, Prins M, Portegies P, Caan MW, Reiss P, Majoie CB, Schmand BA, AGEhIV Cohort Study Group (2015) Multivariate normative comparison, a novel method for more reliably detecting cognitive impairment in HIV infection. AIDS. 29(5):547–557Google Scholar
  35. Tierney SM, Sheppard DP, Kordovski VM, Faytell MP, Avci G, Woods SP (2017) A comparison of the sensitivity, stability, and reliability of three diagnostic schemes for HIV-associated neurocognitive disorders. J Neuro-Oncol 23(3):404–421Google Scholar
  36. Trites RL (1977) Neuropsychological test manual. Royal Ottawa Hospital, OttawaGoogle Scholar
  37. Tymchuk S, Gomez D, Koenig N, Gill MJ, Fujiwara E, Power C (2017) Associations between depressive symptomatology and neurocognitive impairment in HIV/AIDS. Can J Psychiatr 706743717737029Google Scholar
  38. van den Dries LWJ, Wagener MN, Jiskoot LC, Visser M, Robertson KR, Adriani KS, van Gorp ECM (2017) Neurocognitive impairment in a chronically well-suppressed HIV-infected population: the Dutch TREVI Cohort Study. AIDS Patient Care STDs 31(8):329–334CrossRefGoogle Scholar
  39. Vivithanaporn P, Heo G, Gamble J, Krentz HB, Hoke A, Gill MJ, Power C (2010) Neurologic disease burden in treated HIV/AIDS predicts survival: a population-based study. Neurology. 75(13):1150–1158CrossRefGoogle Scholar

Copyright information

© Journal of NeuroVirology, Inc. 2019

Authors and Affiliations

  1. 1.Department of Mathematical and Statistical SciencesUniversity of AlbertaEdmontonCanada
  2. 2.Department of Medicine (Neurology)University of AlbertaEdmontonCanada
  3. 3.Department of MedicineUniversity of CalgaryCalgaryCanada
  4. 4.Department of PsychiatryUniversity of AlbertaEdmontonCanada

Personalised recommendations