Skip to main content
SpringerLink
Log in

Real-Time Feedback of Subjective Affect and Working Memory Load Based on Neurophysiological Activity

  • Conference paper
  • First Online:
HCI International 2021 - Late Breaking Posters (HCII 2021)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1499))

Included in the following conference series:

Abstract

We investigated the effects of feedback on users’ performance during a cognitive task with concurrent emotional distraction. Our aim was to provide participants with insights into their current affective and cognitive state by measuring and decoding brain activity. Therefore, a real-time preprocessing, analyzing, and visualization routine was developed based on electroencephalographic (EEG) data measured during a primary study. To explore users’ behavioral and neurophysiological reactions, error-tolerance as well as possibilities to improve feedback accuracy by the means of feedback-based event-related potentials (ERPs), we provided either legit or inappropriate sham feedback in a second study. The kind of feedback (legit or inappropriate) had only marginal influence on participants’ subsequent performance. On a neuronal level, we did not observe differences in the ERPs evoked by the legit and inappropriate feedback. In qualitative interviews, participants evaluated the feedback as interesting but also sometimes irritating due to odd feedback trials. Our study emphasizes the importance of performance accuracy and transparency towards users regarding the underlying feedback computations.

S. Gado and K. Lingelbach—The Authors contributed equally to this research.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Yu, B., Funk, M., Hu, J., Wang, Q., Feijs, L.: Biofeedback for everyday stress management: a systematic review. Front. ICT 5(23), 1–22 (2018). https://doi.org/10.3389/fict.2018.00023

    Article  Google Scholar 

  2. Dessy, E., Van Puyvelde, M., Mairesse, O., Neyt, X., Pattyn, N.: Cognitive performance enhancement: do biofeedback and neurofeedback work? J. Cogn. Enhancement 2(1), 12–42 (2017). https://doi.org/10.1007/s41465-017-0039-y

    Article  Google Scholar 

  3. Shockley, K.M., Ispas, D., Rossi, M.E., Levine, E.L.: A meta-analytic investigation of the relationship between state affect, discrete emotions, and job performance. Hum. Perform. 25(5), 377–411 (2012). https://doi.org/10.1080/08959285.2012.721832

    Article  Google Scholar 

  4. Niklas, C.D., Dormann, C.: The impact of state affect on job satisfaction. Eur. J. Work Organ. Psychol. 14(4), 367–388 (2005). https://doi.org/10.1080/13594320500348880

    Article  Google Scholar 

  5. Bowling, N.A., Alarcon, G.M., Bragg, C.B., Hartman, M.J.: A meta-analytic examination of the potential correlates and consequences of workload. Work Stress 29(2), 95–113 (2015). https://doi.org/10.1080/02678373.2015.1033037

    Article  Google Scholar 

  6. Moore, M., Shafer, A.T., Bakhtiari, R., Dolcos, F., Singhal, A.: Integration of spatio-temporal dynamics in emotion-cognition interactions: a simultaneous fMRI-ERP investigation using the emotional oddball task. NeuroImage 202, 116078 (2019). https://doi.org/10.1016/j.neuroimage.2019.116078

    Article  Google Scholar 

  7. Maior, H.A., Wilson, M.L., Sharples, S.: Workload alerts - using physiological measures of mental workload to provide feedback during tasks. ACM Trans. Comput.-Hum. Interact. 25(2), 1–25 (2018). https://doi.org/10.1145/3173380

    Article  Google Scholar 

  8. Chen, M., Nikolaidis, S., Soh, H., Hsu, D., Srinivasa, S.: Planning with trust for human-robot collaboration. In: Proceedings of the Anual ACM/IEEE International Conference on Human-Robot Interaction, Chicago, IL, USA 2018, pp. 307–315. Association for Computing Machinery (2018). https://doi.org/10.1145/3171221.3171264

  9. Master, R., et al.: Measurement of trust over time in hybrid inspection systems. Hum. Factors Ergon. Manuf. Serv. Ind. 15(2), 177–196 (2005). https://doi.org/10.1002/hfm.20021

    Article  Google Scholar 

  10. Alder, G.S., Ambrose, M.L.: Towards understanding fairness judgments associated with computer performance monitoring: an integration of the feedback, justice, and monitoring research. Hum. Resour. Manag. Rev. 15(1), 43–67 (2005). https://doi.org/10.1016/j.hrmr.2005.01.001

    Article  Google Scholar 

  11. Ferrez, P.W., Millan, J.d.R.: Error-related EEG potentials generated during simulated brain-computer interaction. IEEE Trans. Biomed. Eng. 55(3), 923–929 (2008). https://doi.org/10.1109/TBME.2007.908083

  12. Mattout, J., Perrin, M., Bertrand, O., Maby, E.: Improving BCI performance through co-adaptation: applications to the P300-speller. Ann. Phys. Rehabil. Med. 58(1), 23–28 (2015). https://doi.org/10.1016/j.rehab.2014.10.006

    Article  Google Scholar 

  13. Pfabigan, D.M., Alexopoulos, J., Bauer, H., Sailer, U.: Manipulation of feedback expectancy and valence induces negative and positive reward prediction error signals manifest in event-related brain potentials. Psychophysiology 48(5), 656–664 (2011). https://doi.org/10.1111/j.1469-8986.2010.01136.x

    Article  Google Scholar 

  14. Enriquez-Geppert, S., Huster, R.J., Herrmann, C.S.: EEG-neurofeedback as a tool to modulate cognition and behaviour: a review tutorial. Front. Hum. Neurosci. 11(51), 1–19 (2017). https://doi.org/10.3389/fnhum.2017.00051

    Article  Google Scholar 

  15. Logemann, H.N.A., Lansbergen, M.M., Van Os, T.W.D.P., Böcker, K.B.E., Kenemans, J.L.: The Effectiveness of EEG-feedback on attention, impulsivity and EEG: a sham feedback controlled study. Neurosci. Lett. 479(1), 49–53 (2010). https://doi.org/10.1016/j.neulet.2010.05.026

    Article  Google Scholar 

  16. Guger, C., Krausz, G., Allison, B., Edlinger, G.: Comparison of dry and gel based electrodes for P300 brain-computer interfaces. Front. Neurosci. 6(60), 1–7 (2012). https://doi.org/10.3389/fnins.2012.00060

    Article  Google Scholar 

  17. Smith, E.E., Reznik, S.J., Stewart, J.L., Allen, J.J.B.: Assessing and conceptualizing frontal EEG asymmetry: an updated primer on recording, processing, analyzing, and interpreting frontal alpha asymmetry. Int. J. Psychophysiol. 111, 98–114 (2017). https://doi.org/10.1016/j.ijpsycho.2016.11.005

    Article  Google Scholar 

  18. Käthner, I., Wriessnegger, S.C., Müller-Putz, G.R., Kübler, A., Halder, S.: Effects of mental workload and fatigue on the P300, alpha and theta band power during operation of an ERP (P300) brain-computer interface. Biol. Psychol. 102, 118–129 (2014). https://doi.org/10.1016/j.biopsycho.2014.07.014

    Article  Google Scholar 

  19. Lee, T.-W., Girolami, M., Sejnowski, T.J.: Independent component analysis using an extended infomax algorithm for mixed Subgaussian and Supergaussian sources. Neural Comput. 11(2), 417–441 (1999). https://doi.org/10.1162/089976699300016719

    Article  Google Scholar 

  20. Gramfort, A., et al.: MNE software for processing MEG and EEG data. NeuroImage 86, 446–460 (2014). https://doi.org/10.1016/j.neuroimage.2013.10.027

    Article  Google Scholar 

  21. Chaumon, M., Bishop, D.V.M., Busch, N.A.: A practical guide to the selection of independent components of the electroencephalogram for artifact correction. J. Neurosci. Methods 250, 47–63 (2015). https://doi.org/10.1016/j.jneumeth.2015.02.025

    Article  Google Scholar 

  22. Maris, E., Oostenveld, R.: Nonparametric statistical testing of EEG- and MEG-data. J. Neurosci. Methods 164(1), 177–190 (2007). https://doi.org/10.1016/j.jneumeth.2007.03.024

    Article  Google Scholar 

  23. Kluger, A.N., DeNisi, A.: The effects of feedback interventions on performance: a historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol. Bull. 119(2), 254–284 (1996). https://doi.org/10.1037/0033-2909.119.2.254

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Fraunhofer Institute for Industrial Engineering IAO, Stuttgart, Germany

    Sabrina Gado, Katharina Lingelbach, Michael Bui & Mathias Vukelić

  2. Department of Psychology, Carl Von Ossietzky University, Oldenburg, Germany

    Katharina Lingelbach & Jochem W. Rieger

Authors
  1. Sabrina Gado
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Katharina Lingelbach
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michael Bui
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jochem W. Rieger
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mathias Vukelić
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Sabrina Gado or Katharina Lingelbach .

Editor information

Editors and Affiliations

  1. University of Crete and Foundation for Research and Technology – Hellas (FORTH), Heraklion, Crete, Greece

    Constantine Stephanidis

  2. Foundation for Research and Technology – Hellas (FORTH), Heraklion, Crete, Greece

    Margherita Antona

  3. Foundation for Research and Technology – Hellas (FORTH), Heraklion, Crete, Greece

    Stavroula Ntoa

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gado, S., Lingelbach, K., Bui, M., Rieger, J.W., Vukelić, M. (2021). Real-Time Feedback of Subjective Affect and Working Memory Load Based on Neurophysiological Activity. In: Stephanidis, C., Antona, M., Ntoa, S. (eds) HCI International 2021 - Late Breaking Posters. HCII 2021. Communications in Computer and Information Science, vol 1499. Springer, Cham. https://doi.org/10.1007/978-3-030-90179-0_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-90179-0_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-90178-3

  • Online ISBN: 978-3-030-90179-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation