Skip to main content
SpringerLink
Log in

Emotion Measurement from Attention Analysis on Imagery in Virtual Reality

  • Conference paper
  • First Online:
Book cover Advances in Affective and Pleasurable Design (AHFE 2019)

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 952))

Included in the following conference series:

Abstract

Measuring human emotion from affective interaction is an important part of computing for mental health applications. Our work refers to non-obtrusive emotion measurement from eye tracking that gets increasingly ubiquitous, e.g., in Virtual Reality (VR) technology. We present a concept and first results of a feasibility study in which affectively weighted imagery of an image database is presented and integrated using discriminative observation, multi-object tracking and video-gaming. We measured attention preference for affective image classes and found relevant correlation with data extracted from a questionnaire on emotional states (MDBF) that would eventually substantiate basic valence classification from eye tracking data. The playful approach using the well-known concentration (pairs) game enables frequent repetition of the measurements in mental health care, therapeutic or pedagogical scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Picard, R.W.: Affective Computing. MIT Press, Cambridge (1997)

    Google Scholar 

  2. Breazeal, C.: The role of expressive behavior for robots that learn from people. Philos. Trans. R. Soc. B 364, 3527–3538 (2009)

    Article  Google Scholar 

  3. Pantic, M., Rothkrantz, L.J.M.: Toward an affect-sensitive multimodal human-computer interaction. Proc. IEEE 91(9), 1370–1390 (2003)

    Article  Google Scholar 

  4. Mandryk, R., Atkins, M.: A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies. Int. J. Hum. Comput. Stud. 65, 329–347 (2007)

    Article  Google Scholar 

  5. Harmon-Jones, E., Lueck, L., Fearn, M., Harmon-Jones, C.: The effect of personal relevance and approach related action expectation on relative left frontal cortical activity. Psychol. Sci. 17(5), 434–440 (2006)

    Article  Google Scholar 

  6. Mauss, I.B., Robinson, M.D.: Measures of emotion: a review. Cogn. Emot. 23, 209–237 (2009)

    Article  Google Scholar 

  7. Schuller, B., Batliner, A.: Computational Paralinguistics: Emotion, Affect and Personality in Speech and Language Processing. Wiley, Hoboken (2013)

    Book  Google Scholar 

  8. Pantic, M.: Machine analysis of facial behavior: naturalistic and dynamic behaviour. Philos. Trans. R. Soc. B 364, 3505–3513 (2009)

    Article  Google Scholar 

  9. Dan-Glauser, E.S., Scherer, K.: The Geneva affective picture database (GAPED): a new 730-picture database focusing on valence ad normative significance. Behav. Res. 43, 468–477 (2011)

    Article  Google Scholar 

  10. Hinz, A., et al.: Norms of the multidimensional mood questionnaire MDBF. Psychother. Psych. Med. 62 (2011)

    Google Scholar 

  11. Sarkany, A., Töser, Z., Verö, A., Lörincz, A., Toyama, T., Toois, E.N., Sonntag, D.: Maintain and improve mental health by smart virtual reality serious games (2015)

    Google Scholar 

  12. Pessoa, L.: How do emotion and motivation direct executive control? Trends Cogn. Sci. 13, 160–166 (2009)

    Article  Google Scholar 

  13. Ochsner, K.N., Gross, J.J.: The cognitive control of emotion. Trends Cogn. Sci. 9, 242–249 (2005)

    Article  Google Scholar 

  14. Gregg, L., Tarrier, N.: Virtual reality in mental health. A review of the literature. Soc. Psychiatry Psychiatr. Epidemiol. 42(5), 343–354 (2007)

    Article  Google Scholar 

  15. Freeman, D., Reeve, S., Robinson, A., Ehlers, A., Clark, D., Spanlang, B., Slater, M.: Virtual reality in the assessment, understanding, and treatment of mental health disorders. Psychol. Med. 47, 2393–2400 (2017)

    Article  Google Scholar 

  16. Tseng, P.H., Cameron, I.G.M., Pari, G., Reynolds, J.N., Munoz, D.P., Itti, L.: High-throughput classification of clinical populations from natural viewing eye movements. J. Neurol. (2012)

    Google Scholar 

  17. Molitor, R.J., Ko, P.C., Ally, B.A.: Eye movements in Alzheimer’s disease. J. Alzheimers Dis. 44, 1–12 (2015)

    Article  Google Scholar 

  18. Kemner, C., Verbaten, M.N., Cuperus, J.M., Camfferman, G., Engeland, H.: Abnormal saccadic eye movements in autistic children. J. Autism Dev. Disord. 28(1), 61–67 (1998)

    Article  Google Scholar 

  19. Armstrong, T., Olatunji, B.O.: Eye tracking of attention in the affective disorders: a meta-analytic review and synthesis. Clin. Psychol. Rev. 32, 704–723 (2012)

    Article  Google Scholar 

  20. Sears, C.R., Thomas, C.L., LeHuquet, J.M., Johnson, J.C.S.: Attentional bias in dysphoria: an eye-tracking study of the allocation and disengagement of attention. Cogn. Emot. 24, 1349–1368 (2010)

    Article  Google Scholar 

  21. Steidmann, D., Ingram, R.E., Siegle, G.J.: Pupil response to negative emotional information in individuals at risk for depression. Cogn. Emot. 24, 480–496 (2010)

    Article  Google Scholar 

  22. Joormann, J., Gotlib, I.: Selective attention to emotional faces following recovery from depression. J. Abnorm. Psychol. 116(1), 80–85 (2007)

    Article  Google Scholar 

  23. Watson, D., Clark, L.A.: The PANAS-X: Manual for the Positive and Negative Affect Schedule Expanded Form. University of Iowa, Cedar Rapids (1999)

    Book  Google Scholar 

  24. Toure-Tillery, M., Fishbach, A.: How to measure motivation: a guide for the experimental social psychologist. Soc. Pers. Psychol. Compass 8, 328–341 (2014)

    Article  Google Scholar 

  25. Finnema, E., Dröes, R.-M., Ettema, T.P., van Tilburg, W.: The effect of integrated emotion-oriented care versus usual care on elderly persons with dementia in the nursing home and on nursing assistants: a randomized clinical trial. Int. J. Geriatr. Psychiatry 20, 330–343 (2005)

    Article  Google Scholar 

  26. Pylyshyn, Z.W., Storm, R.W.: Tracking multiple independent targets: evidence for a parallel tracking mechanism. Spat. Vis. 3(3), 1–19 (1988)

    Article  Google Scholar 

  27. Robinson, P., el Kaliouby, R.: Computation of emotion in man and machines. Philos. Trans. R. Soc. (2009)

    Google Scholar 

  28. Wenzlaff, R.M., Rude, S.S., Taylor, C.J., Stultz, C.H., Sweatt, R.A.: Beneath the veil of thought suppression: attentional bias and depression risk. Cogn. Emot. 15, 435–452 (2001)

    Article  Google Scholar 

  29. Serino, S., Matic, A., Giakoumis, D., Lopez, G., Cipresso, P. (eds.): Pervas. Comp. Parad. M.Health (2015)

    Google Scholar 

  30. Boyle, G.J., Helmes, E., Matthews, G., Carroll, E.I.: Measures of affect dimensions. In: Measures of Personality and Social Psychological Constructs (2015)

    Google Scholar 

  31. Desmet, P.M.A., et al.: Mood measurement with Pick-A-Mood; review of current methods and design of a pictorial self-report scale. J. Des. Res. 14, 241–279 (2016)

    Google Scholar 

  32. Morris, J.D., Bradley, M., Lang, J., Waine, C.: Assessing affective reactions to advertisements with (SAM) the Self-Assessment Manikin. University of Florida (1992)

    Google Scholar 

  33. Lithari, C., et al.: Are females more responsive to emotional stimuli? A neurophysiological study across arousal and valence dimensions. Brain Topogr. 23, 27–40 (2010)

    Article  Google Scholar 

  34. Wadley, G., Calvo, R.A., Torous, J., Czerwinksi, M.: 3rd symposium on computing and mental health: understanding, engaging, and delighting users. In: Proceedings of the 2018 CHI Conference Extended Abstracts on Human Factors in Computing Systems (2018)

    Google Scholar 

Download references

Acknowledgments

The research leading to these results has received funding from the project PLAYTIME of the AAL Programme of the European Union, by the Austrian BMVIT/FFG (No. 857334), by the Styrian Fund for the Future within project SenseCity (No. 8009), as well as by the Austrian BMVIT/FFG by projects AMIGO (No. 865646) and OpenSense (No. 868218).

Author information

Authors and Affiliations

  1. DIGITAL - Institute for Information and Communication Technologies, JOANNEUM RESEARCH Forschungsgesellschaft mbH, Steyrergasse 17, 8010, Graz, Austria

    Lucas Paletta, Amir Dini & Martin Pszeida

Authors
  1. Lucas Paletta
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Amir Dini
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Martin Pszeida
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lucas Paletta .

Editor information

Editors and Affiliations

  1. System Design and Management Research Institute, Keio University, Yokohama, Japan

    Shuichi Fukuda

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Paletta, L., Dini, A., Pszeida, M. (2020). Emotion Measurement from Attention Analysis on Imagery in Virtual Reality. In: Fukuda, S. (eds) Advances in Affective and Pleasurable Design. AHFE 2019. Advances in Intelligent Systems and Computing, vol 952. Springer, Cham. https://doi.org/10.1007/978-3-030-20441-9_2

Download citation

Publish with us

Policies and ethics

Search

Navigation