Skip to main content

Advertisement

SpringerLink
Log in

Now All Together: Overview of Virtual Health Assistants Emulating Face-to-Face Health Interview Experience

  • Technical Contribution
  • Published:
KI - Künstliche Intelligenz Aims and scope Submit manuscript

Abstract

We discuss a large research project aimed at building socially expressive virtual health agents or assistants (VHA) that can deliver brief motivational interventions (BMI) for behavior change, in a communication style that individuals and patients not only accept, but also find emotionally supportive and socially appropriate. Because of their well-defined sequential structure, BMIs lend themselves well to automation, and are adaptable to address a variety of target behaviors, from obesity, to alcohol and drug use, to lack treatment adherence, among others. We discuss the advantages that VHAs provide for the delivery of health interventions. We describe components of our intelligent agent architecture that enables our virtual health agents to dialogue with users in realtime while delivering the appropriate intervention based on the patient’s specific needs at the time. We conclude by identifying open research challenges in developing virtual health agents.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. We use the term evidence-based as most scholars in medicine and healthcare seem to agree that the evidence-based decision-making process integrates (1) best available research evidence, (2) practitioner expertise and other available resources, and (3) the characteristics, needs, values, and preferences of those who will be affected by the intervention [24]

  2. It is important to note that, unlike avatars that are representation of the user in a virtual environment controlled or tele-operated by the user, IVAs are autonomous entities capable of making decisions and of interacting with users

  3. It is important to note that our system can be adapted to any type of brief motivational intervention, e.g. against obesity.

  4. RL raises an individuals’ awareness about their at-risk behavior(s)—an initial important step toward behavior change

  5. We use the SightCorp off-the-shelf face reader for detecting emotional facial expressions

References

  1. Abeyruwan S, Baral R, Yasavur U, Lisetti C, Visser U (2014) Humanoid robots and spoken dialog systems for brief health interventions. In: 2014 AAAI Fall symposium series

  2. Amini R (2015) Learning data-driven models of non-verbal behaviors for building rapport using an intelligent virtual agent. Ph.D. thesis, Florida International University

  3. Amini R, Lisetti C (2013) HapFACS : an open source API / software to generate FACS-based expressions for ECAs animation and for corpus generation. In: The fifth biannual humaine association conference on affective computing and intelligent interaction (ACII 2013). IEEE computer society, Geneva, Switzerland

  4. Babor TF, Grant M (1992) Programme on substance abuse : project on identification and management of alcohol-related problems. report on phase ii, an randomized clinical trial of brief interventions in primary health care

  5. Bavelas JB, Black A, Lemery CR, Mullett J (1987) Empathy and its developement. In: Eisenberg N, Strayer J (eds) Motor mimicry as primitive empathy. Cambridge University Press, Cambridge, UK, pp 317–338

    Google Scholar 

  6. Bickmore T, Gruber A, Picard R (2005) Establishing the computer—patient working alliance in automated health behavior change interventions. Patient Educ Couns 59:21–30

    Article  Google Scholar 

  7. Bohus D, Rudnicky AI (2009) The ravenclaw dialog management framework: architecture and systems. Comput Speech Lang 23(3):332–361

    Article  Google Scholar 

  8. Boukricha H, Wachsmuth I (2011) Empathy-based emotional alignment for a virtual human: a three-step approach. KI-Künstliche Intelligenz 25(3):195–204

    Article  Google Scholar 

  9. Cassell J, Sullivan J, Prevost S, Churchill EF (2000) Embodied conversational agents. Soc Psychol 40(1):26–36

    Google Scholar 

  10. Chartrand TL, Bargh JA (1999) The chameleon effect: the perception-behavior link and social interaction. J Personal Soc Psychol 76(6):893–910

    Article  Google Scholar 

  11. Cooper L, Powe N (2004) Disparities in patient experiences, health care processes, and outcomes: the role of patient-provider racial, ethnic, and language concordance. Health Care 753. http://www.commonwealthfund.org/publications/fund-reports/2004/jul/disparities-in-patient-experiences--health-care-processes--and-outcomes--the-role-of-patient-provide

  12. Cooper LA, Roter DL, Johnson RL, Ford DE, Steinwachs DM, Powe NR (2003) Patient-centered communication, ratings of care, and concordance of patient and physician race. Ann Intern Med 139(11):907–915

    Article  Google Scholar 

  13. Cunningham JA, Humphreys K, Koski-Jannes A (1999) Providing personalized assessment feedback for problem drinking on the Internet. In: Paper presented at the 33rd annual convention of the Association for the advancement of behavior therapy, Toronto

  14. Doherty Y, Hall D, James PT, Roberts SH, Simpson J (2000) Change counselling in diabetes: The development of a training programme for the diabetes team. Patient Edu Couns 40:263–278

    Article  Google Scholar 

  15. Emmons KM, Rollnick S (2001) Motivational interviewing in health care settings. Opportunities and limitations. Am J Prev Med 20(1):68–74

    Article  Google Scholar 

  16. Evers KE, Cummins CO, Prochaska JO, Prochaska JM (2005) Online health behavior and disease management programs: are we ready for them? Are they ready for us? J Med Internet Res 7(3):e27

    Article  Google Scholar 

  17. Ferguson G, Allen J et al (1998) Trips: an integrated intelligent problem-solving assistant. In: Proceedings of the National conference on artificial intelligence, AAAI Press, New York, pp 567–573

  18. Frampton M, Lemon O (2009) Recent research advances in reinforcement learning in spoken dialogue systems. Knowl Eng Rev 24(4):375–408

    Article  Google Scholar 

  19. Hester RK, Squires DD, Delaney HD (2005) The Drinker’s Check-up: 12-month outcomes of a controlled clinical trial of a stand-alone software program for problem drinkers. J Subst Abuse Treat 28(2):159–169

    Article  Google Scholar 

  20. Hester RK, Delaney H (1997) Behavioral self-control program for windows: results of a controlled clinical trial. J Consult Clin Psychol 65:685–693

    Article  Google Scholar 

  21. Hoffman ML (2000) Empathy and moral development: implications for caring and justice. Cambridge University Press, New York, NY

    Book  Google Scholar 

  22. Hojat M (2007) Empathy in patient care: antecedents, development, measurement, and outcomes. Springer, New York

    Google Scholar 

  23. Huang L, Morency LP, Gratch J (2001) Virtual Rapport 2.0. In: International conference on intelligent virtual agents, intelligence, lecture notes in artificial intelligence. Springer, Berlin pp 68–79

  24. Jacobs JA, Jones E, Gabella BA, Spring B, Brownson C (2012) Tools for implementing an evidence-based approach in public health practice the need for evidence-based public health. Prev Chron Dis 9(1):1–9

    Google Scholar 

  25. Lakin JL, Chartrand TL (2003) Using nonconscious behavioral mimicry to create affiliation and rapport. Psychol Sci 14(4):334–339

    Article  Google Scholar 

  26. Lee J, Marsella SC (2009) Learning a model of speaker head nods using gesture corpora. In: Decker, Sichman, Sierra, Castelfranchi (eds.) 8th Int’l Conf. on autonomous agents and multiagent systems (AAMAS 2009)

  27. Lisetti CL, Amini R, Yasavur U, Rishe N (2013) I can help you change! An empathic virtual Agent delivers behavior change health interventions. ACM Trans Manage Info Syst 4(4). doi:10.1145/2544103

  28. Miller WR, Wilbourne PL (2002) Mesa grande: a methodological analysis of clinical trials of treatments for alcohol use disorders. Addiction 97(3):265–277

    Article  Google Scholar 

  29. Miller WR, Rollnick SE (2002) Motivational interviewing. Preparing people to change addictive behavior, 2nd edn. Guilford Press, New York

    Google Scholar 

  30. Mori M (1970) The Uncanny Valley. Energy 7(4):33–35

    Google Scholar 

  31. Morrow T (2012) Virtual health assistants poised to revolutionize healthcare delivery. Am J Manage Care pp 13–14

  32. Moyer A, Finney JW, Swearingen CE, Vergun P (2002) Brief interventions for alcohol problems: a meta-analytic review of controlled investigations in treatment-seeking and non-treatment-seeking populations. Addiction 97(3):279–292

    Article  Google Scholar 

  33. Nass C, Isbister K, Lee E (2000) Truth is beauty: researching embodied conversational agents. In: J. Cassell (ed.) Embodied conversational agents. MIT Press, Cambridge

  34. National Center for Chronic Prevention and Health Promotion (2011) Excessive alcohol use at a glance: addressing a leading risk for death, chronic disease, and Injury. Report Article.

  35. NIAAA: Niaaa alcohol alert no. 66: Brief interventions (2006). http://pubs.niaaa.nih.gov/publications/AA66/AA66.pdf

  36. Norfolk T, Birdi K, Walsh D (2007) The role of empathy in establishing rapport in the consultation: a new model. Med Educ 41(7):690–7

    Article  Google Scholar 

  37. Ochs M, Sadek D, Pelachaud C (2012) A formal model of emotions for an empathic rational dialog agent. Auton Agents Multi-Agent Syst 24(3):410–440

    Article  Google Scholar 

  38. Paek T, Pieraccini R (2008) Automating spoken dialogue management design using machine learning: an industry perspective. Speech commun 50(8):716–729

    Article  Google Scholar 

  39. Pelachaud C (2009) Modelling multimodal expression of emotion in a virtual agent. Philos Trans R Soc Lond B Biol Sci 364(1535):3539–48

    Article  Google Scholar 

  40. Portnoy DB, Scott-Sheldon LAJ, Johnson BT, Carey MP (2008) Computer-delivered interventions for health promotion and behavioral risk reduction: a meta-analysis of 75 randomized controlled trials, 1988–2007. Prev Med 47(1):3–16

    Article  Google Scholar 

  41. Prochaska JO, Velicer WF (1997) The transtheoretical model of health behavior change. Am J Health Promot 12(1):38–48

    Article  Google Scholar 

  42. Rabiner L (1989) A tutorial on hidden Markov models and selected applications in speech recognition. Proc IEEE 77(2):257–286

    Article  Google Scholar 

  43. Reeves B, Nass C, Reeves B, Nass C (1996) The media equation: how people treat computers, television, and new media like real people and places. University of Chicago Press, New York

    Google Scholar 

  44. Resnicow K, Soler R, Braithwaite RL, Ahluwalia JS, Butler J (2000) Cultural sensitivity in substance use prevention. J Community Psychol 28:271–290

    Article  Google Scholar 

  45. Rogers CR (1959) A theory of therapy, personality and interpersonal relationships as developed in the client-centered framework. In: S. Koch (ed) Psychology: the Study of a Science, vol 3, chap 3, McGraw-Hill, New York, pp 184–256

  46. Rollnick S, Miller W (1995) What is motivational interviewing? Behav Cogn Psychother 23:325–334

    Article  Google Scholar 

  47. Servan-Schreiber D (1986) Artificial intelligence and psychiatry. J Nerv Ment Dis 174:191–202

    Article  Google Scholar 

  48. Singh S, Litman D, Kearns M, Walker M (2002) Optimizing dialogue management with reinforcement learning: experiments with the njfun system. J Artif Intell Res 16:105–133

    Google Scholar 

  49. Skinner HA (1994) Computerized lifestyle assessment manual. Multi-Health Systems, Toronto

    Google Scholar 

  50. Spek V, Cuijpers P, Nyklícek I, Riper H, Keyzer J, Pop V (2007) Internet-based cognitive behaviour therapy for symptoms of depression and anxiety: a meta-analysis. Psychol Med 37 (November 2006), 319–328

  51. Squires DD, Hester RK (2004) Using technical innovations in clinical practice: the Drinker’s Check-Up software program. J Clin Psychol 60(2):159–69

    Article  Google Scholar 

  52. Sutton S, Cole R, De Villiers J, Schalkwyk J, Vermeulen P, Macon M, Yan Y, Kaiser E, Rundle B, Shobaki K et al (1998) Universal speech tools: The cslu toolkit. In: Proceedings of the international conference on spoken language processing (ICSLP), Sydney, Australia, pp 3221–3224

  53. Swanson AJ, Pantalon MV, Cohen KR (1999) Motivational interviewing and treatment adherence among psychiatric and dually diagnosed patients. J Nerv Ment Dis 187:630–635

    Article  Google Scholar 

  54. Tickle-Degnen L, Rosenthal R (1990) The nature of rapport and its nonverbal correlates. Psychol Inq 1(4):285–293

    Article  Google Scholar 

  55. Traum D, Larsson S (2003) The information state approach to dialogue management. In: van Kuppevelt J, Smith RW (eds) Current and new directions in discourse and dialogue text, speech and language technology, Kluwer Academic Publishers, p 325–353

  56. Vernon M (2010) A review of computer-based alcohol problem services designed for the general public. J Subst Abuse Treat 38(3):203–211

    Article  Google Scholar 

  57. Walitzer KS, Dermen KH, Connors GJ (1999) Strategies for preparing clients for treatment. A review. Behav Modif 23(1):129–151

    Article  Google Scholar 

  58. WHO World Health Organization (2014) Obesity and overweight fact sheet 311

  59. Williams JD (2008) The best of both worlds: unifying conventional dialog systems and pomdps. In: INTERSPEECH, pp 1173–1176

  60. Wispé L (1987) History of the concept of empathy. In: Einsenberg N, Strayer J (eds) Empathy and its development, vol 2, Cambridge University Press, Cambridge, pp 17–37

    Google Scholar 

  61. Yasavur U, Lisetti C, Rishe N (2014) Let’s talk! speaking virtual counselor offers you a brief intervention. J Multimodal User Interfaces 8:381–398

    Article  Google Scholar 

  62. Yevlahova D, Satur J (2009) Models for individual oral health promotion and their effectiveness: A systematic review. Aust Dent J 54(3):190–197

    Article  Google Scholar 

  63. Young S, Gašić M, Keizer S, Mairesse F, Schatzmann J, Thomson B, Yu K (2010) The hidden information state model: apractical framework for pomdp-based spoken dialogue management. Comput Speech Lang 24(2):150–174

    Article  Google Scholar 

  64. Young S, Gašić M, Thomson B, Williams J (2013) Pomdp-based statistical spoken dialog systems: a review. In Proceedings of the IEEE, 10(5):1160–1179

Download references

Author information

Authors and Affiliations

  1. 11200 SW 8th Street ECS 361, Miami, FL, 33199, USA

    Christine Lisetti, Reza Amini & Ugan Yasavur

Authors
  1. Christine Lisetti
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Reza Amini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ugan Yasavur
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christine Lisetti.

Additional information

Part of the research described in this article was developed with funding from the National Science Foundation, grant numbers HRD-0833093, IIP-1338922, IIP- 1237818, and IIS-1423260.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lisetti, C., Amini, R. & Yasavur, U. Now All Together: Overview of Virtual Health Assistants Emulating Face-to-Face Health Interview Experience. Künstl Intell 29, 161–172 (2015). https://doi.org/10.1007/s13218-015-0357-0

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13218-015-0357-0

Keywords

Search

Navigation