Using fNIRS and EDA to Investigate the Effects of Messaging Related to a Dimensional Theory of Emotion

  • Jan WatsonEmail author
  • Amanda Sargent
  • Yigit Topoglu
  • Hongjun Ye
  • Wenting Zhong
  • Rajneesh Suri
  • Hasan Ayaz
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 953)


Effective techniques for the analysis of messaging strategies is critical as targeted messaging is ubiquitous in society and a major research interest in the fields of psychology, business, marketing and communications. In this study, we investigated the effect of audiovisual messaging on participants’ affective state using a two-dimensional theory of emotion with orthogonal valence and arousal axes. Twenty-four participants were recruited and presented with either a positively framed or negatively framed environmental conservation messaging video. We monitored participants’ finger based electrodermal activity (EDA) as well as prefrontal hemodynamic activity using functional near infrared spectroscopy (fNIRS) during message viewing to attain measures related to neural activity and arousal. Consistent with our expectations, combined results from EDA and prefrontal asymmetry from fNIRS indicate positively framed messaging stimuli was related to higher arousal and higher valence compared to the negatively framed messaging stimuli. Combined brain and body imaging provides a comprehensive assessment and fNIRS + EDA can be used in the future for the neuroergonomic assessment of cognitive and affective state of individuals in real-world environments via wearable sensors.


Functional near infrared spectroscopy Electrodermal activity Affective state assessment Neuroergonomics 


  1. 1.
    Pelletier, E., Sharp, L.G.: Persuasive communication and proenvironmental behaviours: how message tailoring and message framing can improve the integration of behaviours through self-determined motivation. Can. Psychol. 49(3), 210–217 (2008)CrossRefGoogle Scholar
  2. 2.
    Schmid, K.L., Rivers, S.E., Latimer, A.E., Salovey, P.: Targeting or tailoring? Mark. Health Serv. 28(1), 32–37 (2008)Google Scholar
  3. 3.
    Ayaz, H., Onaral, B., Izzetoglu, K., Shewokis, P.A., McKendrick, R., Parasuraman, R.: Continuous monitoring of brain dynamics with functional near infrared spectroscopy as a tool for neuroergonomic research: empirical examples and a technological development. Front. Hum. Neurosci. 7(December), 1–13 (2013)Google Scholar
  4. 4.
    Mehta, R.K., Parasuraman, R.: Neuroergonomics: a review of applications to physical and cognitive work. Front. Hum. Neurosci. 7(December), 1–10 (2013)Google Scholar
  5. 5.
    Ayaz, H., Dehais, F.: Neuroergonomics: The Brain at Work and in Everyday Life. Academic Press, London (2018)Google Scholar
  6. 6.
    Parasuraman, R., Rizzo, M.: Neuroergonomics: The Brain at Work. Oxford University Press, Incorporated, Cary (2006)CrossRefGoogle Scholar
  7. 7.
    Parasuraman, R., Jiang, Y.: Individual differences in cognition, affect, and performance: behavioral, neuroimaging, and molecular genetic approaches. Neuroimage 59(1), 70–82 (2012)CrossRefGoogle Scholar
  8. 8.
    Villegas, J., Matyas, C., Srinivasan, S., Cahyanto, I., Thapa, B., Pennington-Gray, L.: Cognitive and affective responses of Florida tourists after exposure to hurricane warning messages. Nat. Hazards 66(1), 97–116 (2013)CrossRefGoogle Scholar
  9. 9.
    Armitage, C.J., Conner, M.: Efficacy of the theory of planned behaviour: a meta-analytic review. Br. J. Soc. Psychol. 40, 471–499 (2001)CrossRefGoogle Scholar
  10. 10.
    Webb, T.L., Sheeran, P.: Does changing behavioral intentions engender behavior change? A meta-analysis of the experimental evidence. Psychol. Bull. 132(2), 249–268 (2006)CrossRefGoogle Scholar
  11. 11.
    Cooper, N., Tompson, S., O’Donnell, M.B., Emily, B.F.: Brain activity in self- and value-related regions in response to online antismoking messages predicts behavior change. J. Media Psychol. 27(3), 93–109 (2015)CrossRefGoogle Scholar
  12. 12.
    Berkman, E.T., Falk, E.B.: Beyond brain mapping. Curr. Dir. Psychol. Sci. 22(1), 45–50 (2013)CrossRefGoogle Scholar
  13. 13.
    Leger, P.-M., Riedl, R., vom Brocke, J.: Emotions and ERP information sourcing: the moderating role of expertise. Ind. Manag. Data Syst. 114(3), 456–471 (2014)CrossRefGoogle Scholar
  14. 14.
    Miller, G.A.: How we think about cognition, emotion and biology in psychopathology. In: Psychophysiology, vol. 46, pp. S12–S12. Blackwell Publishers, Malden (2009)Google Scholar
  15. 15.
    Lang, P.J.: The emotion probe: studies of motivation and attention. Am. Psychol. 50(5), 372–385 (1995)CrossRefGoogle Scholar
  16. 16.
    Ravaja, N., Saari, T., Salminen, M., Laarni, J., Kallinen, K.: Phasic emotional reactions to video game events: a psychophysiological investigation. Media Psychol. 8(4), 343–367 (2006)CrossRefGoogle Scholar
  17. 17.
    Carver, C.S.: Approach, avoidance, and the self-regulation of affect and action. Motiv. Emot. 30(2), 105–110 (2006)CrossRefGoogle Scholar
  18. 18.
    Davidson, R.J.: Cerebral asymmetry, emotion and affective style. In: Davidson, R.J., Hugdahl, K. (eds.) Brain Asymmetry. The MIT Press, Cambridge (1995)Google Scholar
  19. 19.
    Palmiero, M., Piccardi, L.: Frontal EEG asymmetry of mood: a mini-review. Front. Behav. Neurosci. 11(November), 1–8 (2017)Google Scholar
  20. 20.
    Zhao, Z., Glover, G.: Hemispheric asymmetry for emotional stimuli. NeuroReport 9(14), 3233–3239 (1998)CrossRefGoogle Scholar
  21. 21.
    Lang, P.J., Greenwald, M.K., Bradley, M.M., Hamm, A.O.: Looking at pictures: affective, facial, visceral, and behavioral reactions. Psychophysiology 30(3), 261–273 (1993)CrossRefGoogle Scholar
  22. 22.
    Di Domenico, S.I., Le, A., Liu, Y., Ayaz, H., Fournier, M.A.: Basic psychological needs and neurophysiological responsiveness to decisional conflict: an event-related potential study of integrative self processes. Cogn. Affect. Behav. Neurosci. 16(5), 848–865 (2016)CrossRefGoogle Scholar
  23. 23.
    Davidson, R.J.: What does the prefrontal cortex ‘do’ in affect: perspectives on frontal EEG asymmetry research. Biol. psychol. 67(1–2), 219 (2004)CrossRefGoogle Scholar
  24. 24.
    Vecchiato, G., et al.: Spectral EEG frontal asymmetries correlate with the experienced pleasantness of TV commercial advertisements. Med. Biol. Eng. Comput. 49(5), 579–583 (2011)CrossRefGoogle Scholar
  25. 25.
    Ohme, R., Reykowska, D., Wiener, D., Choromanska, A.: Application of frontal EEG asymmetry to advertising research. J. Econ. Psychol. 31(5), 785–793 (2010)CrossRefGoogle Scholar
  26. 26.
    Vecchiato, G., et al.: Changes in brain activity during the observation of TV commercials by using EEG, GSR and HR measurements. Brain Topogr. 23(2), 165–179 (2010)CrossRefGoogle Scholar
  27. 27.
    Balconi, M., Molteni, E.: Past and future of near-infrared spectroscopy in studies of emotion and social neuroscience. J. Cogn. Psychol. 28(2), 129–146 (2016)CrossRefGoogle Scholar
  28. 28.
    Aday, J., Rizer, W., Carlson, J.M.: Neural mechanisms of emotions and affect. In: Emotions and Affect in Human Factors and Human-Computer Interaction, pp. 27–87. Elsevier, London (2017)CrossRefGoogle Scholar
  29. 29.
    Yuksel, B.F., Oleson, K.B., Chang, R., Jacob, R.J.K.: Detecting and adapting to users’ cognitive and affective state to develop intelligent musical interfaces. In: New Directions in Music and Human-Computer Interaction, pp. 163–177. Springer, Cham (2019)CrossRefGoogle Scholar
  30. 30.
    Herrmann, M.J., Ehlis, A.-C., Fallgatter, A.J.: Prefrontal activation through task requirements of emotional induction measured with NIRS. Biol. Psychol. 64(3), 255–263 (2003)CrossRefGoogle Scholar
  31. 31.
    Harmon-Jones, E., Gable, P.A., Peterson, C.K.: The role of asymmetric frontal cortical activity in emotion-related phenomena: a review and update. Biol. Psychol. 84(3), 451–462 (2010)CrossRefGoogle Scholar
  32. 32.
    Benedek, M., Kaernbach, C.: A continuous measure of phasic electrodermal activity. J. Neurosci. Methods 190(1), 80–91 (2010)CrossRefGoogle Scholar
  33. 33.
    Greco, A., Valenza, G., Citi, L., Scilingo, E.P.: Arousal and valence recognition of affective sounds based on electrodermal activity. IEEE Sens. J. 17(3), 716–725 (2017)CrossRefGoogle Scholar
  34. 34.
    Poh, M., Member, S., Swenson, N.C., Picard, R.W.: A wearable sensor for unobtrusive, long-term assessment of electrodermal activity. IEEE Trans. Biomed. Eng. 57(5), 1243–1252 (2010)CrossRefGoogle Scholar
  35. 35.
    Bradley, M.M., Lang, P.J.: Affective reactions to acoustic stimuli. Psychophysiology 37(2), S0048577200990012 (2000)CrossRefGoogle Scholar
  36. 36.
    Franceschini, M.A., Boas, D.A.: Noninvasive measurement of neuronal activity with near-infrared optical imaging. Neuroimage 21(1), 372–386 (2004)CrossRefGoogle Scholar
  37. 37.
    Izzetoglu, K., Bunce, S., Izzetoglu, M., Onaral, B., Pourrezaei, K.: Functional near-infrared neuroimaging. IEEE Trans. Neural Syst. Rehabil. Eng. 13(2), 5333–5336 (2004)Google Scholar
  38. 38.
    Plichta, M.M., et al.: Auditory cortex activation is modulated by emotion: a functional near-infrared spectroscopy (fNIRS) study. Neuroimage 55(3), 1200–1207 (2011)CrossRefGoogle Scholar
  39. 39.
    Yang, H., et al.: Gender difference in hemodynamic responses of prefrontal area to emotional stress by near-infrared spectroscopy. Behav. Brain Res. 178(1), 172–176 (2007)CrossRefGoogle Scholar
  40. 40.
    Leon-Carrion, J., et al.: Differential time course and intensity of PFC activation for men and women in response to emotional stimuli: a functional near-infrared spectroscopy (fNIRS) study. Neurosci. Lett. 403(1–2), 90–95 (2006)CrossRefGoogle Scholar
  41. 41.
    Kreplin, U., Fairclough, S.H.: Activation of the rostromedial prefrontal cortex during the experience of positive emotion in the context of esthetic experience. An fNIRS study. Front. Hum. Neurosci. 7(December), 1–7 (2013)Google Scholar
  42. 42.
    Di Domenico, S.I., Rodrigo, A.H., Ayaz, H., Fournier, M.A., Ruocco, A.C.: Decision-making conflict and the neural efficiency hypothesis of intelligence: a functional near-infrared spectroscopy investigation. Neuroimage 109, 307–317 (2015)CrossRefGoogle Scholar
  43. 43.
    Ayaz, H., Shewokis, P.A., Curtin, A., Izzetoglu, M., Izzetoglu, K., Onaral, B.: Using MazeSuite and functional near infrared spectroscopy to study learning in spatial navigation. J. Vis. Exp. 56, 1–13 (2011)Google Scholar
  44. 44.
    McKendrick, R., Ayaz, H., Olmstead, R., Parasuraman, R.: Enhancing dual-task performance with verbal and spatial working memory training: continuous monitoring of cerebral hemodynamics with NIRS. Neuroimage 85, 1014–1026 (2014)CrossRefGoogle Scholar
  45. 45.
    Liu, Y., et al.: Measuring speaker–listener neural coupling with functional near infrared spectroscopy. Sci. Rep. 7(1), 43293 (2017)CrossRefGoogle Scholar
  46. 46.
    Khalfa, S., Isabelle, P., Jean-Pierre, B., Manon, R.: Event-related skin conductance responses to musical emotions in humans. Neurosci. Lett. 328(2), 145–149 (2002)CrossRefGoogle Scholar
  47. 47.
    Balconi, M., Grippa, E., Vanutelli, M.E.: Resting lateralized activity predicts the cortical response and appraisal of emotions: an fNIRS study. Soc. Cogn. Affect. Neurosci. 10(12), 1607–1614 (2015)CrossRefGoogle Scholar
  48. 48.
    Tuscan, L.-A., Herbert, J.D., Forman, E.M., Juarascio, A.S., Izzetoglu, M., Schultheis, M.: Exploring frontal asymmetry using functional near-infrared spectroscopy: a preliminary study of the effects of social anxiety during interaction and performance tasks. Brain Imaging Behav. 7(2), 140–153 (2013)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Jan Watson
    • 1
    Email author
  • Amanda Sargent
    • 1
  • Yigit Topoglu
    • 1
  • Hongjun Ye
    • 2
  • Wenting Zhong
    • 2
  • Rajneesh Suri
    • 2
    • 3
  • Hasan Ayaz
    • 1
    • 3
    • 4
    • 5
  1. 1.School of Biomedical Engineering, Science and Health SystemsDrexel UniversityPhiladelphiaUSA
  2. 2.LeBow College of BusinessDrexel UniversityPhiladelphiaUSA
  3. 3.Drexel Business Solutions InstituteDrexel UniversityPhiladelphiaUSA
  4. 4.Department of Family and Community HealthUniversity of PennsylvaniaPhiladelphiaUSA
  5. 5.Center for Injury Research and PreventionChildren’s Hospital of PhiladelphiaPhiladelphiaUSA

Personalised recommendations