Skip to main content
SpringerLink
Log in

Physiological Computing: Interfacing with the Human Nervous System

  • Chapter
  • First Online:
Sensing Emotions

Part of the book series: Philips Research Book Series ((PRBS,volume 12))

Abstract

This chapter describes the physiological computing paradigm where electrophysiological changes from the human nervous system are used to interface with a computer system in real time. Physiological computing systems are categorized into five categories: muscle interfaces, brain-computer interfaces, biofeedback, biocybernetic adaptation and ambulatory monitoring. The differences and similarities of each system are described. The chapter also discusses a number of fundamental issues for the design of physiological computing system, these include: the inference between physiology and behaviour, how the system represents behaviour, the concept of the biocybernetic control loop and ethical issues.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Aarts, E.: Ambient intelligence: a multimedia perspective. IEEE Multimed. 11(1), 12–19 (2004)

    Article  Google Scholar 

  • Ahn, H., Teeters, A., Wang, A, Breazeal, C., Picard, R.W.: Stoop to conquer: posture and affect interact to influence computer users’ persistence. Paper presented at the second international conference on affective computing and intelligent interaction, Lisbon, Portugal (2007)

    Google Scholar 

  • Allison, B., Graimann, B., Graser, A.: Why use a BCI if you are healthy? Paper presented at the ACE workshop – Brainplay’07: brain-computer interfaces and games, Salzburg (2007)

    Google Scholar 

  • Allison, B.Z., Wolpaw, E.W., Wolpaw, J.R.: Brain-computer interface systems: progress and prospects. Expert Rev. Med. Devices 4(4), 463–474 (2007)

    Article  Google Scholar 

  • Baber, C., Haniff, D.J., Woolley, S.I.: Contrasting paradigms for the development of wearable computers. IBM Syst. J. 38(4), 551–565 (1999)

    Article  Google Scholar 

  • Bartlett, M., Littlewort, G., Fasel, I., Morvellan, J.R.: Real time face detection and facial expression recognition: development and application to human-computer interaction. Paper presented at the computer vision and pattern recognition for human-computer interaction, Vancouver (2003)

    Google Scholar 

  • Brin, D.: The Transparent Society: Will Technology Force Us to Choose between Privacy and Freedom. Perseus Books, New York, NY (1999)

    Google Scholar 

  • Burleson, W., Picard, R.W.: Affective agents: sustaining motivation to learn through failure and a state of “stuck”. Paper presented at the workshop on social and emotional intelligence in learning environments, Maceio, Alagoas, Brazil (2004)

    Google Scholar 

  • Cacioppo, J.T., Berntson, G.G., Malarkey, W., Kiecolt-Glaser, J.K., Sheridan, J.F., Poehlmann, K. et al.: Autonomic, neuroendocrine and immune responses to psychological stress: the reactivity hypothesis. Ann. N Y Acad. Sci. 840, 664–673 (1998)

    Article  Google Scholar 

  • Cacioppo, J.T., Tassinary, L.G.: Inferring psychological significance from physiological signals. Am. Psychol. 45(1), 16–28 (1990)

    Article  Google Scholar 

  • Cacioppo, J.T., Tassinary, L.G., Berntson, G.G.: Handbook of Psychophysiology. Cambridge University Press, New York, NY (2000a)

    Google Scholar 

  • Cacioppo, J.T., Tassinary, L.G., Berntson, G.G.: Psychophysiological science. In: Cacioppo, J.T., Tassinary, L.G., Berntson, G.G. (eds.) Handbook of Psychophysiology, pp. 3–26. Cambridge University Press, Cambridge (2000b)

    Google Scholar 

  • Chin, C.A., Barreto, A., Cremades, J.G., Adjouadi, M.: Integrated electromyogram and eye-gaze tracking cursor control system for computer users with motor disabilities. J. Rehabil. Res. Dev. 45(1), 161–174 (2008)

    Article  Google Scholar 

  • Coan, J.A., Allen, J.J.B.: Frontal EEG asymmetry and the behavioural activation and inhibition systems. Psychophysiology 40, 106–114 (2003)

    Article  Google Scholar 

  • Dekker, A., Champion, E.: Please biofeed the zombies: enhancing the gameplay and display of a horror game using biofeedback. Paper presented at the DiGRA, Tokyo (2007)

    Google Scholar 

  • Ebner-Priemer, U.W., Trill, T.J.: Ambulatory assessment: an innovative and promising approach for clinical psychology. Eur. Psychol. 14(2), 109–119 (2009)

    Article  Google Scholar 

  • Fairclough, S.H.: Psychophysiological inference and physiological computer games. Paper presented at the ACE workshop – Brainplay’07: brain-computer interfaces and games, Salzburg (2007)

    Google Scholar 

  • Fairclough, S.H.: BCI and physiological computing: Similarities, differences and intuitive control. Paper presented at the workshop on BCI and computer games: CHI’08 (2008)

    Google Scholar 

  • Fairclough, S.H.: Fundamentals of physiological computing. Interact. Comput. 21, 133–145 (2009)

    Article  Google Scholar 

  • Fairclough, S.H., Venables, L.: Psychophysiological candidates for biocybernetic control of adaptive automation. In: de Waard, D., Brookhuis, K.A., Weikert, C.M. (eds.) Human Factors in Design, pp. 177–189. Shaker, Maastricht, The Netherlands (2004)

    Google Scholar 

  • Fairclough, S.H., Venables, L., Tattersall, A.: The use of autonomic measures for biocybernetic adaptation. Psychophysiology 42(S1), S25 (2006)

    Google Scholar 

  • Freeman, F.G., Mikulka, P.J., Scerbo, M.W., Scott, L.: An evaluation of an adaptive automation system using a cognitive vigilance task. Biol. Psychol. 67, 283–297 (2004)

    Article  Google Scholar 

  • Gevins, A., Smith, M.E., Leong, H., McEvoy, L., Whitfield, S., Du, R. et al.: Monitoring working memory load during computer-based tasks with EEG pattern recognition models. Hum. Factors 40(1), 79–91 (1998)

    Article  Google Scholar 

  • Gilleade, K.M., Dix, A.: Using frustration in the design of adaptive videogame. Paper presented at the advances in computer entertainment technology (2004)

    Google Scholar 

  • Gilleade, K.M., Dix, A., Allanson, J.: Affective videogames and modes of affective gaming: Assist me, challenge me, emote me. Paper presented at the proceedings of DiGRA (2005)

    Google Scholar 

  • Grauman, K., Betke, M., Gips, J., Bradski, G.R.: Communication via eye blinks – detection and duration analysis in real time. Paper presented at the IEEE computer society conference on Computer Vision and Pattern Recognition (CVPR’01) (2001)

    Google Scholar 

  • Grimes, D., Tan, D.S., Hudson, S.E., Shenoy, P., Rao, R.P.N.: Feasibility and pragmatics of classifying working memory load with an electroencelograph. Paper presented at the CHI (2008)

    Google Scholar 

  • Guger, C., Edlinger, A.G., Harkam, W., Niedermayer, I., Pfurtscheller, G.: How many people are able to operate an EEG-based brain-computer interface (BCI)? IEEE Trans. Neural Syst. Rehabil. Eng. 11(2), 145–147 (2003)

    Article  Google Scholar 

  • Hancock, P.A.: Teleology for technology. In: Parasuraman, R., Mouloua, M. (eds.) Automation and Human Performance, pp. 461–497. Erlbaum, Hillsdale (1996)

    Google Scholar 

  • Hancock, P.A., Pepe, A.A., Murphy, L.L.: Hedonomics: The power of positive and pleasurable ergonomics. Ergon. Des. 13(1), 8–14 (2005)

    Article  Google Scholar 

  • Hancock, P.A., Szalma, J.L.: The future of neuroergonomics. Theor. Issues Ergon. Sci. 44(1–2), 238–249 (2003)

    Article  Google Scholar 

  • Healey, J., Picard, R.W.: Digital processing of affective signals. Paper presented at the proceedings of ICASSP, Seatle, WA (1997)

    Google Scholar 

  • Helander, M.G., Tham, M.P.: Hedonomics – affective human factors design. Ergonomics 46(13/14), 1269–1272 (2003)

    Article  Google Scholar 

  • Hettinger, L.J., Branco, P., Encarnaco, L.M., Bonato, P.: Neuroadaptive technologies: applying neuroergonomics to the design of advanced interfaces. Theor. Issues Ergon. Sci. 4(1–2), 220–237 (2003)

    Article  Google Scholar 

  • Huang, C.N., Chen, C.H., Chung, H.Y.: Application of facial electromyography in computer mouse access for people with disabilities. Disabil. Rehabil. 28(4), 231–237 (2006)

    Article  Google Scholar 

  • Kapoor, A., Burleson, W., Picard, R.W.: Automatic prediction of frustration. Int. J. Hum. Comput. Stud. 65, 724–736 (2007)

    Article  Google Scholar 

  • Karwowski, W.: Symvatology: the science of artifact-human compatibility. Theor. Issues Ergon. Sci. 1(1), 76–91 (2000)

    Article  Google Scholar 

  • Kelly, K.: The technium. Available from http://www.kk.org/thetechnium/archives/2006/02/symmetrical_and.php (2006)

  • Klein, G., Woods, D.D., Bradshaw, J.M., Hoffman, R.R., Feltovich, P.J.: Ten challenges for making automation a “team player” in joint human-agent activity. IEEE Intell. Syst. 19(6), 91–95 (2004)

    Article  Google Scholar 

  • Kosmack Vaara, E., Hook, K., Tholander, J.: Mirror bodily experiences over time. Paper presented at the CHI, Boston (2009)

    Google Scholar 

  • Kulic, D., Croft, E.: Anxiety detection during human-robot interaction. Paper presented at the conference on intelligent robots and systems, Edmonton, Canada (2005)

    Google Scholar 

  • Kulic, D., Croft, E.: Estimating robot induced affective state using Hidden Markov Models. Paper presented at the IEEE international symposium on robot and human interactive communication (2006)

    Google Scholar 

  • Lisetti, C.L., Nasoz, F.: Using noninvasive wearable computers to recognize human emotions from physiological signals. EURASIP J. Appl. Signal Process. 11, 1672–1687 (2004)

    Google Scholar 

  • Lisetti, C.L., Nasoz, F., LeRouge, C., Ozyer, O., Alvarez, K.: Developing multimodal intelligent affective interfaces for tele-home health care. Int. J. Hum. Comput. Stud. 59, 245–255 (2003)

    Article  Google Scholar 

  • Liu, C., Rani, P., Sarkar, N.: An empirical study of machine learning techniques for affect recognition in human-robot interaction. Paper presented at the IEEE conference on intelligent robots and systems (2005)

    Google Scholar 

  • Mandryk, R.L., Atkins, M.S.: A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies. Int. J. Hum. Comput. Stud. 65, 329–347 (2007)

    Article  Google Scholar 

  • Mandryk, R.L., Inkpen, K.M., Calvert, T.W.: Using psychophysiological techniques to measure user experience with entertainment technologies. Behav. Inform. Technol. 25(2), 141–158 (2006)

    Article  Google Scholar 

  • McFetridge-Durdle, J.A., Routledge, F.S., Parry, M.J.E, Dean, C.R.T, Tucker, B.: Ambulatory impedance cardiography in hypertension: a validation study. Eur. J. Cardiovasc. Nurs. 7, 204–213 (2008)

    Article  Google Scholar 

  • Merrill, D., Maes, P.: Augmenting looking, pointing and reaching gestures to enhance the searching and browsing of physical objects. Paper presented at the 5th international conference on pervasive computing (Pervasive’07), Toronto (2007)

    Google Scholar 

  • Microsoft Project Natal: http://www.xbox.com/en-US/live/projectnatal/

  • Milenkovic, A., Otto, C., Jovanov, E.: Wireless sensor networks for personal health monitoring: Issues and an implementation. Comput. Commun. 29, 2521–2533 (2006)

    Article  Google Scholar 

  • Miller, C.A.: Trust in adaptive automation: the role of etiquette in tuning trust via analogic and affective methods. Paper presented at the first international conference on augmented cognition, Las Vegas, 22nd–27th July 2005

    Google Scholar 

  • Morris, M.E.: Technologies for heart and mind: new directions in embedded assessment. Intel. Technol. J. 11(1), 67–74 (2007)

    Google Scholar 

  • Morris, M., Guilak, F.: Mobile heart health: project highlight. IEEE Pervasive Comput. 8(2), 57–61 (2009)

    Article  Google Scholar 

  • Murata, A.: Eye gaze input versus mouse: cursor control as a function of age. Int. J. Hum. Comput. Interact. 21(1), 1–14 (2006)

    Google Scholar 

  • Naik, G.R., Kumar, D.K., Arjunan, S.P.: Multi modal gesture identification for HCI using surface EMG. Paper presented at the 12th international conference on entertainment and media in the ubiquitous era, Tampere (2008)

    Google Scholar 

  • Norman, D.A.: The Design of Future Things. Basic Books, New York, NY (2007)

    Google Scholar 

  • O’Donnell, R.D., Eggemeier, FT: Workload assessment methodology. In: Boff, K., Kaufman, L., Thomas, J. (eds.) Handbook of Human Perception and Performance, vol. 2, pp. 42.41–42.49. Wiley, New York, NY (1986)

    Google Scholar 

  • Picard, R.W.: Affective Computing. MIT Press, Cambridge (1997)

    Google Scholar 

  • Picard, R.W., Healey, J.: Affective wearables. Personal Technol. 1, 231–240 (1997)

    Article  Google Scholar 

  • Picard, R.W., Klein, J.: Computers that recognise and respond to user emotion: theoretical and practical implications. Interact. Comput. 14, 141–169 (2002)

    Article  Google Scholar 

  • Picard, R.W., Vyzas, E., Healey, J: Towards machine emotional intelligence: analysis of affective physiological state. IEEE Trans. Pattern Anal. Mach. Intell. 23(10), 1175–1191 (2001)

    Article  Google Scholar 

  • Pope, A.T., Bogart, E.H., Bartolome, D.S.: Biocybernetic system evaluates indices of operator engagement in automated task. Biol. Psychol. 40, 187–195 (1995)

    Article  Google Scholar 

  • Prinzel, L.J., Freeman, F.G., Scerbo, M.W., Mikulka, P.J., Pope, A.T.: A closed-loop system for examining psychophysiological measures for adaptive task allocation. Int. J. Aviat. Psychol. 10(4), 393–410 (2000)

    Article  Google Scholar 

  • Prinzel, L.J., Freeman, F.G., Scerbo, M.W., Mikulka, P.J., Pope, A.T.: Effects of a psychophysiological system for adaptive automation on performance, workload, and the event-related potential P300 component. Hum. Factors. 45(4), 601–613 (2003)

    Article  Google Scholar 

  • Prinzel, L.J., Parasuraman, R., Freeman, F.G., Scerbo, M.W., Mikulka, P.J., Pope, A.T.: Three experiments examining the use of electroencephalogram, event-related potentials, and heart-rate variability for real-time human-centred adaptive automation design (No. NASA/TP-2003-212442): NASA (2003)

    Google Scholar 

  • Prinzel, L.J., Pope, A.T., Freeman, F.G.: Physiological self-regulation and adaptive automation. Int. J. Aviat. Psychol. 12(2), 179–196 (2002)

    Article  Google Scholar 

  • Rani, P., Sarkar, N.: Operator engagement detection for robot behaviour adaptation. Int. J. Adv. Robot Syst. 4(1), 1–12 (2007)

    Google Scholar 

  • Rani, P., Sarkar, N., Liu, C.: Maintaining optimal challenge in computer games through real-time physiological feedback. Paper presented at the 11th human-computer interaction international, Las Vegas (2005)

    Google Scholar 

  • Rani, P., Sims, J., Brackin, R., Sarkar, N.: Online stress detection using psychophysiological signal for implicit human-robot cooperation. Robotica 20(6), 673–686 (2002)

    Article  Google Scholar 

  • Rashidi, P., Cook, D.J.: Keeping the resident in the loop: Adapting the smart home to the user. IEEE Trans. Syst. Man Cybern. – Part A: Syst. Hum. 38, 949–959 (2009)

    Google Scholar 

  • Reynolds, C., Picard, R.W.: Affective sensors, privacy and ethical contracts. Paper presented at the CHI’2005, Vienna, Austria (2005a)

    Google Scholar 

  • Reynolds, C., Picard, R.W.: Evaluation of affective computing systems from a dimensional metaethical position. Paper presented at the First Augmented Cognition Conference, Las Vegas (2005b)

    Google Scholar 

  • Ryan, R.M., Rigby, C.S., Przybylski, A: The motivational pull of video games: a self-determination approach. Motiv. Emot. 30, 347–363 (2006)

    Google Scholar 

  • Saponas, T.S., Tan, D.S., Morris, D., Balakrishnan, R.: Demonstrating the feasibility of using forearm electromyography for muscle-computer interfaces. Paper presented at the CHI (2008)

    Google Scholar 

  • Scerbo, M.W., Freeman, F.G., Mikulka, P.J.: A brain-based system for adaptive automation. Theor. Issues Ergon. Sci. 4(1–2), 200–219 (2003)

    Article  Google Scholar 

  • Scheirer, J., Fernandez, R., Klein, J., Picard, R.W.: Frustrating the user on purpose: a step toward building an affective computer. Interact. Comput. 14, 93–188 (2002)

    Article  Google Scholar 

  • Schwartz, M.S., Andrasik, F.: Biofeedback: A Practioners Guide. Guildford Press, New York, NY (2003)

    Google Scholar 

  • Tecce, J.J., Gips, J., Olivieri, C.P., Pok, L.J., Consiglio, M.R.: Eye movement control of computer functions. Int. J. Psychophysiol. 29, 319–325 (1998)

    Article  Google Scholar 

  • Teller, A.: A platform for wearable physiological computing. Interact. Comput. 16, 917–937 (2004)

    Article  Google Scholar 

  • Wachsmuth, S., Wrede, S., Hanheide, M.: Coordinating interactive vision behaviours for cognitive assistance. Comput. Vis. Image Underst. 108, 135–149 (2007)

    Article  Google Scholar 

  • Wickens, C.D.: Multiple resources and performance prediction. Theor. Issues Ergon. Sci. 3, 150–177 (2002)

    Article  Google Scholar 

  • Wilson, G.F., Russell, C.A.: Operator functional state classification using multiple psychophysiological features in an air traffic control task. Hum. Factors 45(3), 381–389 (2003)

    Article  Google Scholar 

  • Wilson, G.F., Russell, C.A.: Performance enhancement in an uninhabited air vehicle task using psychophysiologically determined adaptive aiding. Hum. Factors 49(6), 1005–1018 (2007)

    Article  Google Scholar 

  • Wolpaw, J.R., Birbaumer, N., Heetderks, W.J., McFarland, D.J., Peckham, P.H., Schalk, G. et al.: Brain-computer interface technology: a review of the first international meeting. IEEE Trans. Rehabil. Eng. 8, 164–173 (2000)

    Article  Google Scholar 

  • Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., Vaughan, T.M.: Brain-computer interfaces for communication and control. Clin. Neurophysiol. 113, 767–791 (2002)

    Article  Google Scholar 

  • Wolpaw, J.R., McFarland, D.J.: Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. Proc. Natl. Acad. Sci. USA 101, 17849–17854 (2004)

    Article  Google Scholar 

  • Yannakakis, G.N., Hallam, J., Hautop Lund, H.: Entertainment capture through heart rate activity on physical interactive playgrounds. User Model. User-Adapt. Interact. 18, 207–243 (2007)

    Article  Google Scholar 

  • Zander, T.O., Jatzev, S.: Detecting affective covert user states with passive brain-computer interfaces. Paper presented at the Affective Computing and Intelligent Interaction (ACII’09) (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Natural Sciences and Psychology, Liverpool John Moores University, Liverpool, UK

    Stephen H. Fairclough

Authors
  1. Stephen H. Fairclough
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stephen H. Fairclough .

Editor information

Editors and Affiliations

  1. Philips Research, High Tech Campus 34, Eindhoven, 5656 AE, Netherlands

    Joyce Westerink

  2. Philips Research, High Tech Campus 34, Eindhoven, 5656 AE, Netherlands

    Martijn Krans

  3. Philips Research, High Tech Campus 34, Eindhoven, 5656 AE, Netherlands

    Martin Ouwerkerk

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer Science+Business Media B.V.

About this chapter

Cite this chapter

Fairclough, S.H. (2010). Physiological Computing: Interfacing with the Human Nervous System. In: Westerink, J., Krans, M., Ouwerkerk, M. (eds) Sensing Emotions. Philips Research Book Series, vol 12. Springer, Dordrecht. https://doi.org/10.1007/978-90-481-3258-4_1

Download citation

  • DOI: https://doi.org/10.1007/978-90-481-3258-4_1

  • Published:

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-90-481-3257-7

  • Online ISBN: 978-90-481-3258-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation