KI - Künstliche Intelligenz

, Volume 29, Issue 2, pp 161–172 | Cite as

Now All Together: Overview of Virtual Health Assistants Emulating Face-to-Face Health Interview Experience

  • Christine Lisetti
  • Reza Amini
  • Ugan Yasavur
Technical Contribution


We discuss a large research project aimed at building socially expressive virtual health agents or assistants (VHA) that can deliver brief motivational interventions (BMI) for behavior change, in a communication style that individuals and patients not only accept, but also find emotionally supportive and socially appropriate. Because of their well-defined sequential structure, BMIs lend themselves well to automation, and are adaptable to address a variety of target behaviors, from obesity, to alcohol and drug use, to lack treatment adherence, among others. We discuss the advantages that VHAs provide for the delivery of health interventions. We describe components of our intelligent agent architecture that enables our virtual health agents to dialogue with users in realtime while delivering the appropriate intervention based on the patient’s specific needs at the time. We conclude by identifying open research challenges in developing virtual health agents.


Intelligent virtual health agents Embodied conversational agents (ECA) Computational models of empathy and rapport Spoken dialog systems for Health Markov decision process (MDP) Hidden Markov models (HMM) Motivational interviewing Brief interventions Behavior change 


  1. 1.
    Abeyruwan S, Baral R, Yasavur U, Lisetti C, Visser U (2014) Humanoid robots and spoken dialog systems for brief health interventions. In: 2014 AAAI Fall symposium seriesGoogle Scholar
  2. 2.
    Amini R (2015) Learning data-driven models of non-verbal behaviors for building rapport using an intelligent virtual agent. Ph.D. thesis, Florida International UniversityGoogle Scholar
  3. 3.
    Amini R, Lisetti C (2013) HapFACS : an open source API / software to generate FACS-based expressions for ECAs animation and for corpus generation. In: The fifth biannual humaine association conference on affective computing and intelligent interaction (ACII 2013). IEEE computer society, Geneva, SwitzerlandGoogle Scholar
  4. 4.
    Babor TF, Grant M (1992) Programme on substance abuse : project on identification and management of alcohol-related problems. report on phase ii, an randomized clinical trial of brief interventions in primary health careGoogle Scholar
  5. 5.
    Bavelas JB, Black A, Lemery CR, Mullett J (1987) Empathy and its developement. In: Eisenberg N, Strayer J (eds) Motor mimicry as primitive empathy. Cambridge University Press, Cambridge, UK, pp 317–338Google Scholar
  6. 6.
    Bickmore T, Gruber A, Picard R (2005) Establishing the computer—patient working alliance in automated health behavior change interventions. Patient Educ Couns 59:21–30CrossRefGoogle Scholar
  7. 7.
    Bohus D, Rudnicky AI (2009) The ravenclaw dialog management framework: architecture and systems. Comput Speech Lang 23(3):332–361CrossRefGoogle Scholar
  8. 8.
    Boukricha H, Wachsmuth I (2011) Empathy-based emotional alignment for a virtual human: a three-step approach. KI-Künstliche Intelligenz 25(3):195–204CrossRefGoogle Scholar
  9. 9.
    Cassell J, Sullivan J, Prevost S, Churchill EF (2000) Embodied conversational agents. Soc Psychol 40(1):26–36Google Scholar
  10. 10.
    Chartrand TL, Bargh JA (1999) The chameleon effect: the perception-behavior link and social interaction. J Personal Soc Psychol 76(6):893–910CrossRefGoogle Scholar
  11. 11.
    Cooper L, Powe N (2004) Disparities in patient experiences, health care processes, and outcomes: the role of patient-provider racial, ethnic, and language concordance. Health Care 753.
  12. 12.
    Cooper LA, Roter DL, Johnson RL, Ford DE, Steinwachs DM, Powe NR (2003) Patient-centered communication, ratings of care, and concordance of patient and physician race. Ann Intern Med 139(11):907–915CrossRefGoogle Scholar
  13. 13.
    Cunningham JA, Humphreys K, Koski-Jannes A (1999) Providing personalized assessment feedback for problem drinking on the Internet. In: Paper presented at the 33rd annual convention of the Association for the advancement of behavior therapy, TorontoGoogle Scholar
  14. 14.
    Doherty Y, Hall D, James PT, Roberts SH, Simpson J (2000) Change counselling in diabetes: The development of a training programme for the diabetes team. Patient Edu Couns 40:263–278CrossRefGoogle Scholar
  15. 15.
    Emmons KM, Rollnick S (2001) Motivational interviewing in health care settings. Opportunities and limitations. Am J Prev Med 20(1):68–74CrossRefGoogle Scholar
  16. 16.
    Evers KE, Cummins CO, Prochaska JO, Prochaska JM (2005) Online health behavior and disease management programs: are we ready for them? Are they ready for us? J Med Internet Res 7(3):e27CrossRefGoogle Scholar
  17. 17.
    Ferguson G, Allen J et al (1998) Trips: an integrated intelligent problem-solving assistant. In: Proceedings of the National conference on artificial intelligence, AAAI Press, New York, pp 567–573Google Scholar
  18. 18.
    Frampton M, Lemon O (2009) Recent research advances in reinforcement learning in spoken dialogue systems. Knowl Eng Rev 24(4):375–408CrossRefGoogle Scholar
  19. 19.
    Hester RK, Squires DD, Delaney HD (2005) The Drinker’s Check-up: 12-month outcomes of a controlled clinical trial of a stand-alone software program for problem drinkers. J Subst Abuse Treat 28(2):159–169CrossRefGoogle Scholar
  20. 20.
    Hester RK, Delaney H (1997) Behavioral self-control program for windows: results of a controlled clinical trial. J Consult Clin Psychol 65:685–693CrossRefGoogle Scholar
  21. 21.
    Hoffman ML (2000) Empathy and moral development: implications for caring and justice. Cambridge University Press, New York, NYCrossRefGoogle Scholar
  22. 22.
    Hojat M (2007) Empathy in patient care: antecedents, development, measurement, and outcomes. Springer, New YorkGoogle Scholar
  23. 23.
    Huang L, Morency LP, Gratch J (2001) Virtual Rapport 2.0. In: International conference on intelligent virtual agents, intelligence, lecture notes in artificial intelligence. Springer, Berlin pp 68–79Google Scholar
  24. 24.
    Jacobs JA, Jones E, Gabella BA, Spring B, Brownson C (2012) Tools for implementing an evidence-based approach in public health practice the need for evidence-based public health. Prev Chron Dis 9(1):1–9Google Scholar
  25. 25.
    Lakin JL, Chartrand TL (2003) Using nonconscious behavioral mimicry to create affiliation and rapport. Psychol Sci 14(4):334–339CrossRefGoogle Scholar
  26. 26.
    Lee J, Marsella SC (2009) Learning a model of speaker head nods using gesture corpora. In: Decker, Sichman, Sierra, Castelfranchi (eds.) 8th Int’l Conf. on autonomous agents and multiagent systems (AAMAS 2009)Google Scholar
  27. 27.
    Lisetti CL, Amini R, Yasavur U, Rishe N (2013) I can help you change! An empathic virtual Agent delivers behavior change health interventions. ACM Trans Manage Info Syst 4(4). doi: 10.1145/2544103
  28. 28.
    Miller WR, Wilbourne PL (2002) Mesa grande: a methodological analysis of clinical trials of treatments for alcohol use disorders. Addiction 97(3):265–277CrossRefGoogle Scholar
  29. 29.
    Miller WR, Rollnick SE (2002) Motivational interviewing. Preparing people to change addictive behavior, 2nd edn. Guilford Press, New YorkGoogle Scholar
  30. 30.
    Mori M (1970) The Uncanny Valley. Energy 7(4):33–35Google Scholar
  31. 31.
    Morrow T (2012) Virtual health assistants poised to revolutionize healthcare delivery. Am J Manage Care pp 13–14Google Scholar
  32. 32.
    Moyer A, Finney JW, Swearingen CE, Vergun P (2002) Brief interventions for alcohol problems: a meta-analytic review of controlled investigations in treatment-seeking and non-treatment-seeking populations. Addiction 97(3):279–292CrossRefGoogle Scholar
  33. 33.
    Nass C, Isbister K, Lee E (2000) Truth is beauty: researching embodied conversational agents. In: J. Cassell (ed.) Embodied conversational agents. MIT Press, CambridgeGoogle Scholar
  34. 34.
    National Center for Chronic Prevention and Health Promotion (2011) Excessive alcohol use at a glance: addressing a leading risk for death, chronic disease, and Injury. Report Article.Google Scholar
  35. 35.
    NIAAA: Niaaa alcohol alert no. 66: Brief interventions (2006).
  36. 36.
    Norfolk T, Birdi K, Walsh D (2007) The role of empathy in establishing rapport in the consultation: a new model. Med Educ 41(7):690–7CrossRefGoogle Scholar
  37. 37.
    Ochs M, Sadek D, Pelachaud C (2012) A formal model of emotions for an empathic rational dialog agent. Auton Agents Multi-Agent Syst 24(3):410–440CrossRefGoogle Scholar
  38. 38.
    Paek T, Pieraccini R (2008) Automating spoken dialogue management design using machine learning: an industry perspective. Speech commun 50(8):716–729CrossRefGoogle Scholar
  39. 39.
    Pelachaud C (2009) Modelling multimodal expression of emotion in a virtual agent. Philos Trans R Soc Lond B Biol Sci 364(1535):3539–48CrossRefGoogle Scholar
  40. 40.
    Portnoy DB, Scott-Sheldon LAJ, Johnson BT, Carey MP (2008) Computer-delivered interventions for health promotion and behavioral risk reduction: a meta-analysis of 75 randomized controlled trials, 1988–2007. Prev Med 47(1):3–16CrossRefGoogle Scholar
  41. 41.
    Prochaska JO, Velicer WF (1997) The transtheoretical model of health behavior change. Am J Health Promot 12(1):38–48CrossRefGoogle Scholar
  42. 42.
    Rabiner L (1989) A tutorial on hidden Markov models and selected applications in speech recognition. Proc IEEE 77(2):257–286CrossRefGoogle Scholar
  43. 43.
    Reeves B, Nass C, Reeves B, Nass C (1996) The media equation: how people treat computers, television, and new media like real people and places. University of Chicago Press, New YorkGoogle Scholar
  44. 44.
    Resnicow K, Soler R, Braithwaite RL, Ahluwalia JS, Butler J (2000) Cultural sensitivity in substance use prevention. J Community Psychol 28:271–290CrossRefGoogle Scholar
  45. 45.
    Rogers CR (1959) A theory of therapy, personality and interpersonal relationships as developed in the client-centered framework. In: S. Koch (ed) Psychology: the Study of a Science, vol 3, chap 3, McGraw-Hill, New York, pp 184–256Google Scholar
  46. 46.
    Rollnick S, Miller W (1995) What is motivational interviewing? Behav Cogn Psychother 23:325–334CrossRefGoogle Scholar
  47. 47.
    Servan-Schreiber D (1986) Artificial intelligence and psychiatry. J Nerv Ment Dis 174:191–202CrossRefGoogle Scholar
  48. 48.
    Singh S, Litman D, Kearns M, Walker M (2002) Optimizing dialogue management with reinforcement learning: experiments with the njfun system. J Artif Intell Res 16:105–133Google Scholar
  49. 49.
    Skinner HA (1994) Computerized lifestyle assessment manual. Multi-Health Systems, TorontoGoogle Scholar
  50. 50.
    Spek V, Cuijpers P, Nyklícek I, Riper H, Keyzer J, Pop V (2007) Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med 37 (November 2006), 319–328Google Scholar
  51. 51.
    Squires DD, Hester RK (2004) Using technical innovations in clinical practice: the Drinker’s Check-Up software program. J Clin Psychol 60(2):159–69CrossRefGoogle Scholar
  52. 52.
    Sutton S, Cole R, De Villiers J, Schalkwyk J, Vermeulen P, Macon M, Yan Y, Kaiser E, Rundle B, Shobaki K et al (1998) Universal speech tools: The cslu toolkit. In: Proceedings of the international conference on spoken language processing (ICSLP), Sydney, Australia, pp 3221–3224Google Scholar
  53. 53.
    Swanson AJ, Pantalon MV, Cohen KR (1999) Motivational interviewing and treatment adherence among psychiatric and dually diagnosed patients. J Nerv Ment Dis 187:630–635CrossRefGoogle Scholar
  54. 54.
    Tickle-Degnen L, Rosenthal R (1990) The nature of rapport and its nonverbal correlates. Psychol Inq 1(4):285–293CrossRefGoogle Scholar
  55. 55.
    Traum D, Larsson S (2003) The information state approach to dialogue management. In: van Kuppevelt J, Smith RW (eds) Current and new directions in discourse and dialogue text, speech and language technology, Kluwer Academic Publishers, p 325–353Google Scholar
  56. 56.
    Vernon M (2010) A review of computer-based alcohol problem services designed for the general public. J Subst Abuse Treat 38(3):203–211CrossRefGoogle Scholar
  57. 57.
    Walitzer KS, Dermen KH, Connors GJ (1999) Strategies for preparing clients for treatment. A review. Behav Modif 23(1):129–151CrossRefGoogle Scholar
  58. 58.
    WHO World Health Organization (2014) Obesity and overweight fact sheet 311Google Scholar
  59. 59.
    Williams JD (2008) The best of both worlds: unifying conventional dialog systems and pomdps. In: INTERSPEECH, pp 1173–1176Google Scholar
  60. 60.
    Wispé L (1987) History of the concept of empathy. In: Einsenberg N, Strayer J (eds) Empathy and its development, vol 2, Cambridge University Press, Cambridge, pp 17–37Google Scholar
  61. 61.
    Yasavur U, Lisetti C, Rishe N (2014) Let’s talk! speaking virtual counselor offers you a brief intervention. J Multimodal User Interfaces 8:381–398CrossRefGoogle Scholar
  62. 62.
    Yevlahova D, Satur J (2009) Models for individual oral health promotion and their effectiveness: A systematic review. Aust Dent J 54(3):190–197CrossRefGoogle Scholar
  63. 63.
    Young S, Gašić M, Keizer S, Mairesse F, Schatzmann J, Thomson B, Yu K (2010) The hidden information state model: apractical framework for pomdp-based spoken dialogue management. Comput Speech Lang 24(2):150–174CrossRefGoogle Scholar
  64. 64.
    Young S, Gašić M, Thomson B, Williams J (2013) Pomdp-based statistical spoken dialog systems: a review. In Proceedings of the IEEE, 10(5):1160–1179Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2015

Authors and Affiliations

  1. 1.MiamiUSA

Personalised recommendations