VICA, a visual counseling agent for emotional distress

  • Yoshitaka Sakurai
  • Yukino Ikegami
  • Motoki Sakai
  • Hiroshi Fujikawa
  • Setsuo Tsuruta
  • Avelino J. Gonzalez
  • Eriko Sakurai
  • Ernesto Damiani
  • Andrea Kutics
  • Rainer Knauf
  • Fulvio Frati
Original Research


We present VICA, a Visual Counseling Agent designed to create an engaging multimedia face-to-face interaction. VICA is a human-friendly agent equipped with high-performance voice conversation designed to help psychologically stressed users, to offload their emotional burden. Such users specifically include non-computer-savvy elderly persons or clients. Our agent builds replies exploiting interlocutor’s utterances expressing such as wishes, obstacles, emotions, etc. Statements asking for confirmation, details, emotional summary, or relations among such expressions are added to the utterances. We claim that VICA is suitable for positive counseling scenarios where multimedia specifically high-performance voice communication is instrumental for even the old or digital divided users to continue dialogue towards their self-awareness. To prove this claim, VICA’s effect is evaluated with respect to a previous text-based counseling agent CRECA and ELIZA including its successors. An experiment involving 14 subjects shows VICA effects as follows: (i) the dialogue continuation (CPS: Conversation-turns Per Session) of VICA for the older half (age > 40) substantially improved 53% to CRECA and 71% to ELIZA. (ii) VICA’s capability to foster peace of mind and other positive feelings was assessed with a very high score of 5 or 6 mostly, out of 7 stages of the Likert scale, again by the older. Compared on average, such capability of VICA for the older is 5.14 while CRECA (all subjects are young students, age < 25) is 4.50, ELIZA is 3.50, and the best of ELIZA’s successors for the older (> 25) is 4.41.


Conversational agent Visual counseling agent Avatar Voice conversation Dialogue continuation Self-awareness 



This work was supported by JSPS KAKENHI Grant Numbers JP15K00349, JP15K00382 and by Artificial Intelligence Research Promotion Foundation. We thank the students of the Distributed Intelligent Systems Lab, Tokyo Denki University Japan and of the Machine Learning Systems Lab, Meiji University Japan for their dedicated help to VICA implementation.


  1. Anisetti M, Bellandi V, Damiani E, Jeon G, Jeong J (2007) Full Controllable Face Detection System Architecture for Robotic Vision. Proceedings 2007 Third International IEEE Conference on Signal-Image Technologies and Internet-Based System. pp. 748–756.
  2. Asay TP, Lambert MJ (1966) The Empirical Case for the Common Factors in Therapy: Quantitative Findings. The HEAT & SOUL of CHANGE, pp. 23–55, 1966.
  3. Ashwell T, Elam J (2017) How accurately can the Google Web Speech API recognize and transcribe Japanese L2 English learners’ oral production? Jalt Call Journal 13(1):59–76Google Scholar
  4. DeVault D et al (2014) SimSensei Kiosk: A virtual human interviewer for healthcare decision support. Proceedings of the 2014 international conference on Autonomous agents and multi-agent systems, pp. 1061–1068Google Scholar
  5. FaceGen (2017) FaceGen Modeller: 3D Face Generator (accessed 26 June, 2018).
  6. Floridi L, Taddeo M, Turilli M (2009) Turing’s Imitation Game: Still an Impossible Challenge for All Machines and Some Judges - An Evaluation of the 2008 Loebner Contest. Mind Mach 19(1):145–150. CrossRefGoogle Scholar
  7. Gonzalez AJ, Tsuruta S, Sakurai Y, Nguyen J, Takada K, Uchida K (2010) Using Contexts to Supervise a Collaborative Process. Artificial Intelligence for Engineering Design. Anal Manufact 25(1):25–40. Google Scholar
  8. Guangbing Y, Nian-Shing C, Kinshuk SE, Anderson T, Wen D (2013) The effectiveness of automatic text summarization in mobile learning contexts. Comput Educ 68:233–243. CrossRefGoogle Scholar
  9. Gupta V, Lehal GS (2010) A survey of text summarization extractive techniques. Journal of Emerging Technologies in Web Intelligence 2(3):258–268. CrossRefGoogle Scholar
  10. Han S, Lee K, Lee D, Lee GG (2013) Counseling Dialog System with 5W1H Extraction. Proceeding of the SIGDIAL 2013 Conference, pp. 349–353.
  11. Hung V, Gonzalez A, De Mara R (2009) Towards a Context-Based Dialog Management Layer for Expert Systems. International Conference on Information, Process, and Knowledge Management, Cancun, 2009, pp. 60–65.
  12. Ivey AE, Packard NG, Ivey MB (2006) Basic Attending Skills, Fourth edn. Microtraning Associates, Ann Arbor, pp 93–97Google Scholar
  13. Kennedy A, Kazantseva A, Inkpen D, Szpakowicz S (2012) Getting emotional about news summarization. Advances in Artificial Intelligence, pp. 121–132.
  14. Likert R (1932) A Technique for the Measurement of Attitudes. Archives of Psychology 22(140):1–55.
  15. Mental Health Foundation (2018) Stress: Are we coping? London: Mental Health Foundation. (accessed 30 December, 2018).
  16. Ministry of Health, Labour and Welfare (2008) General condition of mental health caring and smoking countermeasure consequence (Japanese). Accessed 26 June 2018
  17. Powell K (1966) Richard Rogers: Complete Works - Volume 3. Phaidon Press, LondonGoogle Scholar
  18. Ptaszynski, Michal et al (2009) A system for affect analysis of utterances in Japanese supported with web mining. Journal of Japan Society for Fuzzy Theory Intelligent Informatics 21:2: 194–213CrossRefGoogle Scholar
  19. Rizzo A, Talbot T (2016) Chap. 18: Virtual Reality Standardized Patients for Clinical Training. The Digital Patient: Advancing Healthcare, Research, and Education. John Wiley & Sons, Hoboken. Google Scholar
  20. Rogers CR (1957) The necessary and sufficient conditions of therapeutic personality change. Journal of Consulting Psychology 21(2):95–103. CrossRefGoogle Scholar
  21. Savickas ML (2011) Career Counseling. American Psychological Association, WashingtonGoogle Scholar
  22. Shah H, Warwick K, Vallverde J, Wu D (2016) Can Machines Talk? Comparison of ELIZA with Modern Dialogue Systems. Computers Human Behavior 58(C):278–295. CrossRefGoogle Scholar
  23. Shinozaki T, Yamamoto Y, Tsuruta S (2013) Context-based counselor agent for software development ecosystem. Computing 97(1):3–28. CrossRefGoogle Scholar
  24. Shum H-Y, He X, Li D (2018) From Eliza to XiaoIce: challenges and opportunities with social chatbots Frontiers of Information Technology & Electronic Engineering (ISSN 2095–9184, monthly) Chinese Academy of Engineering (CAE) and Zhejiang University, co-published co-published by Springer & Zhejiang University Press January 2018, Volume 19, Issue 1, pp 10–26Google Scholar
  25. Wallace RS (2003) The elements of AIML style. ALICE AI Foundation. Retrieved on July 15, 2018 from
  26. Weissman J, Russell D, Jay M, Beasley JM, Malaspina D, Pegus C (2017) Disparities in Health Care Utilization and Functional Limitations Among Adults With Serious Psychological Distress, 2006–2014. Psychiatric Services 68(7):653–659. CrossRefGoogle Scholar
  27. Weizenbaum J (1966) ELIZA - A Computer Program For the Study of Natural Language Communication Between Man and Machine. Commun ACM 9(1):36–45. CrossRefGoogle Scholar
  28. Weizenbaum J (1976) Computer Power and Human Reason: From Judgment to Calculation. W. H. Freeman & Co., New YorkGoogle Scholar
  29. Yamamoto Y, Shinozaki T, Tsuruta S, Damiani E, Knauf R (2016) Equipping a Context Respectful Counseling Agent with a Human-like Voice Synthesizer. Proceedings of 2016 World Automation Congress (WAC), pp. 1–6.

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  • Yoshitaka Sakurai
    • 1
  • Yukino Ikegami
    • 2
  • Motoki Sakai
    • 3
  • Hiroshi Fujikawa
    • 3
  • Setsuo Tsuruta
    • 3
  • Avelino J. Gonzalez
    • 4
  • Eriko Sakurai
    • 5
  • Ernesto Damiani
    • 6
    • 7
  • Andrea Kutics
    • 8
  • Rainer Knauf
    • 9
  • Fulvio Frati
    • 10
  1. 1.School of Interdisciplinary Mathematical SciencesMeiji UniversityTokyoJapan
  2. 2.IO IncTokyoJapan
  3. 3.School of Information EnvironmentTokyo Denki UniversityInzaiJapan
  4. 4.School of Engineering and Computer ScienceUniversity of Central FloridaOrlandoUSA
  5. 5.Faculty of Service ManagementBunri University of HospitalitySayamaJapan
  6. 6.Center for Cyber-Physical SystemsKhalifa UniversityAbu DhabiUAE
  7. 7.Computer Science DepartmentUniversità degli Studi di MilanoMilanItaly
  8. 8.Department of Natural SciencesInternational Christian UniversityTokyoJapan
  9. 9.Department of Computer Science and AutomationTechnische Universität IlmenauIlmenauGermany
  10. 10.Computer Science DepartmentUniversità degli Studi di MilanoCremaItaly

Personalised recommendations