Using Paralinguistic Cues in Speech to Recognise Emotions in Older Car Drivers

  • Christian Jones
  • Ing-Marie Jonsson
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4868)


Interactive speech based systems are moving into the car since speech interactions are considered less detrimental to the driver than interactions with a display. The introduction of in-car speech-based interactions highlights the potential influence of linguistic and paralinguistic cues such as emotion. Emotions direct and focus people’s attention on objects and situations, and affects performance, judgment and risk-taking. All of these properties are crucial for driving where the smallest slip-up can have grave repercussions. Emotional cues in a car-voice, paired with the emotional state of the driver, have been found to influence driving performance. This initiated the design of an in-car driver emotion detection and response system. Results show that the in-car system can recognise and track changes in the emotional state of the driver. This study considers older drivers who often feel both unsafe and insecure due to concerns about declining abilities and in particular vision.


Automotive control In-car systems Paralinguistic cues Affective computing Emotion recognition Speech interaction 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lunenfeld, H.: Human Factor Considerations of Motorist Navigation and Information Systems. In: Proc. of Vehicle Navigation and Information Systems, pp. 35–42 (1989)Google Scholar
  2. 2.
    Strayer, D., Johnston, W.: Driven to Distraction: Dual-Task Studies of Simulated Driving and Conversing on a Cellular Telephone. Psychological Science 12, 462–466 (2001)CrossRefGoogle Scholar
  3. 3.
    Brave, S., Nass, C.: Emotion in Human-Computer Interaction. In: Jacko, J., Sears, A. (eds.) Handbook of Human-Computer Interaction, pp. 251–271. Lawrence Erlbaum Associates, Mahwah (2002)Google Scholar
  4. 4.
    Pinker, S.: The Language Instinct. W. Morrow and Company, New York (1994)Google Scholar
  5. 5.
    Gross, J.J.: Antecedent- and Response-Focused Emotion Regulation: Divergent Consequences for Experience, Expression, and Physiology. Journal of Personality and Social Psychology 74, 224–237 (1998)CrossRefGoogle Scholar
  6. 6.
    Hirt, E.R., Melton, R.J., McDonald, H.E., Harackiewicz, J.M.: Processing Goals, Task Interest, and the Mood-Performance Relationship: A Mediational Analysis. Journal of Personality and Social Psychology 71, 245–261 (1996)CrossRefGoogle Scholar
  7. 7.
    Isen, A.M.: Positive Affect and Decision Making. In: Lewis, M., Haviland-Jones, J.M. (eds.) Handbook of Emotions, pp. 417–435. The Guilford Press, New York (2000)Google Scholar
  8. 8.
    McGehee, D., Lee, J., Rizzo, M., Bateman, K.: Examination of Older Driver Steering Adaptation on a High Performance Driving Simulator. In: Proceedings of International Driving Symposium on Human Factors in Driver Assessment, Training, and Vehicle Design (2001)Google Scholar
  9. 9.
    Nemoto, H., Yanagishima, T., Taguchi, M.: Effect of Physical Changes in Aging on Driving Performance, Nissan Research Center, Vehicle Research Laboratory. In: Proceeding of First International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design, pp. 131–136 (2001)Google Scholar
  10. 10.
    Ponds, R., Brouwer, W., van Wolffelaar, P.: Age Differences in Divided Attention in a Simulated Driving Task. Journal of Gerontology 43, 151–156 (1988)Google Scholar
  11. 11.
    Johansson, K.: Older Automobile Drivers: Medical Aspects, Doctoral Dissertation, Karolinska Institutet, Stockholm (1997)Google Scholar
  12. 12.
    Owens, J., Lehman., R.: The Effects of Age and Distraction on Reaction Time in a Driving Simulator. Journal of Vision 2(7), 632a (2002)CrossRefGoogle Scholar
  13. 13.
    Nass, C., Jonsson, I.-M., Harris, H., Reaves, B., Endo, J., Brave, S., Takayama, L.: Improving Automotive Safety by Pairing Driver Emotion and Car Voice Emotion. In: CHI 2005 Extended Abstracts on Human factors in Computing Systems, Portland, USA, pp. 1973–1976 (2005)Google Scholar
  14. 14.
    Jones, C.M., Jonsson, I.-M.: Performance Analysis of Acoustic Emotion Recognition for In-Car Conversational Interfaces. In: Proceedings of HCI International, Beijing, China (2007)Google Scholar
  15. 15.
    Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., Taylor, J.: Emotion Recognition in Human-Computer Interaction. IEEE Signal Processing Magazine, 32–80 (January 2001)Google Scholar
  16. 16.
    Brave, S.: Agents that Care: Investigating the Effects of Orientation of Emotion Exhibited by an Embodied Computer Agent. Doctoral Dissertation. Communication. Stanford University, Stanford, CA (2003)Google Scholar
  17. 17.
    Picard, R.W.: Affective Computing. MIT Press, Cambridge (1997)Google Scholar
  18. 18.
    Mullennix, J.W., Bihon, T., Bricklemyer, J., Gaston, J., Keener, J.M.: Effects of Variation in Emotional Tone of Voice on Speech Perception. Language and Speech 45, 228–255 (2002)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2008

Authors and Affiliations

  • Christian Jones
    • 1
  • Ing-Marie Jonsson
    • 2
  1. 1.University of the Sunshine CoastQueenslandAustralia
  2. 2.Department of CommunicationStanford UniversityUSA

Personalised recommendations