Skip to main content
SpringerLink
Log in
Book cover

International Conference on Physiological Computing Systems

International Conference on Physiological Computing Systems

International Conference on Physiological Computing Systems

PhyCS 2016, PhyCS 2017, PhyCS 2018: Physiological Computing Systems pp 218–237Cite as

Integrating Biocybernetic Adaptation in Virtual Reality Training Concentration and Calmness in Target Shooting

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 10057))

Abstract

Training military readiness can significantly reduce potentially avoidable mistakes in real life situations. Virtual Reality (VR) has been widely used to provide a controlled and immersive medium for training both trainees’ physical and cognitive skills. Despite the tremendous advances in VR-based training for military personnel, the attention has been mainly paid on improving simulation’s realism through hardware tools and enhancing graphics and data input paradigms, rather than augmenting the human-computer interaction. Biocybernetic adaptation is a technique from the physiological computing field that allows creating real-time modulations based on detected human states indicated by psychophysiological responses. Although very sophisticated, the creation of biocybernetic loops has been mainly confined to research laboratories and very complex and invasive setups. Moreover, the combination of VR applications and biocybernetic adaptation has rarely been pursued beyond exploratory experiments. The Biocyber Physical System (BioPhyS) for military training in VR constitutes the first fully integrated, distributed and replicable VR simulator that is biocybernetically modulated. BioPhyS uses neurophysiological and cardiovascular measurements recorded from wearable sensors to detect calmness and cognitive readiness states to create dynamic changes in a VR target shooting simulator. The design process, psychophysiological modeling, and biocybernetic loop technology integration are shown, describing a pilot study carried out with a group of non-military participants. We highlight the software elements used for the VR-biocybernetic integration, and the psychophysiological model created for the real-time system as well as the timeline used to develop the functional prototype. We conclude this paper with a set of guidelines for developing meaningful physiological adaptations in VR applications.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    https://sites.google.com/view/physio2games.

  2. 2.

    http://developer.choosemuse.com/tools/mac-tools/muselab.

  3. 3.

    https://developer.polar.com/wiki/H6,_H7,_H10_and_OH1_Heart_rate_sensors.

  4. 4.

    http://developer.choosemuse.com/tools/available-data.

References

  1. Anthes, C., García-Hernández, R.J., Wiedemann, M., Kranzlmüller, D.: State of the art of virtual reality technology (2016)

    Google Scholar 

  2. Jerald, J.: The VR Book: Human-Centered Design for Virtual Reality. Morgan & Claypool, New York (2015)

    Book  Google Scholar 

  3. Kosunen, I., Salminen, M., Järvelä, S., Ruonala, A., Ravaja, N., Jacucci, G.: RelaWorld: neuroadaptive and immersive virtual reality meditation system. In: Proceedings of the 21st International Conference on Intelligent User Interfaces, pp. 208–217. ACM (2016)

    Google Scholar 

  4. Sra, M., Xu, X., Maes, P.: BreathVR: leveraging breathing as a directly controlled interface for virtual reality games. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, p. 340. ACM (2018)

    Google Scholar 

  5. Muñoz, J.E., Paulino, T., Vasanth, H., Baras, K.: PhysioVR: a novel mobile virtual reality framework for physiological computing. In: 2016 IEEE 18th International Conference on e-Health Networking, Applications and Services (Healthcom), pp. 1–6. IEEE (2016)

    Google Scholar 

  6. Pope, A.T., Stephens, C.L., Gilleade, K.: Biocybernetic adaptation as biofeedback training method. In: Fairclough, S.H., Gilleade, K. (eds.) Advances in Physiological Computing. HIS, pp. 91–115. Springer, London (2014). https://doi.org/10.1007/978-1-4471-6392-3_5

    Chapter  Google Scholar 

  7. Fairclough, S.H.: Fundamentals of physiological computing. Interact. Comput. 21, 133–145 (2009)

    Article  Google Scholar 

  8. Novak, D., Mihelj, M., Munih, M.: A survey of methods for data fusion and system adaptation using autonomic nervous system responses in physiological computing. Interact. Comput. 24, 154–172 (2012)

    Article  Google Scholar 

  9. Fairclough, S.H.: Physiological computing and intelligent adaptation. In: Emotions and Affect in Human Factors and Human-Computer Interaction, pp. 539–556. Elsevier (2017)

    Google Scholar 

  10. Burdea, G.C., Coiffet, P.: Virtual Reality technology. Wiley, Hoboken (2003)

    Book  Google Scholar 

  11. Kennedy, R.S., Stanney, K.M., Lawson, B.D.: Capability of virtual environments to meet military requirements. Essex Corp., Orlando, FL (2000)

    Google Scholar 

  12. Mead, C.: War Play: Video Games and the Future of Armed Conflict. Houghton Mifflin Harcourt, Boston (2013)

    Google Scholar 

  13. Roy, M.J., Rizzo, A., Difede, J., Rothbaum, B.O.: Virtual reality exposure therapy for PTSD. In: Complementary and Alternative Medicine for PTSD, p. 271 (2016)

    Google Scholar 

  14. Wood, D.P., et al.: Cost effectiveness of virtual reality graded exposure therapy with physiological monitoring for the treatment of combat related post traumatic stress disorder. Stud. Health Technol. Inform. 144, 223–229 (2009)

    Google Scholar 

  15. Ćosić, K., Popović, S., Kukolja, D., Horvat, M., Dropuljić, B.: Physiology-driven adaptive virtual reality stimulation for prevention and treatment of stress related disorders. Cyberpsychology Behav. Soc. Netw. 13, 73–78 (2010)

    Article  Google Scholar 

  16. Blacker, K.J., Hamilton, J., Roush, G. et al.: Cognitive training for military application: a review of the literature and practical guide. J. Cogn. Enhanc. 3, 30 (2019). https://doi.org/10.1007/s41465-018-0076-1

    Article  Google Scholar 

  17. Lele, A.: Virtual reality and its military utility. J. Ambient. Intell. Hum. Comput. 4, 17–26 (2013)

    Article  Google Scholar 

  18. Palsson, O.S., Harris Sr, R.L., Pope, A.T.: Method and apparatus for encouraging physiological self-regulation through modulation of an operator’s control input to a video game or training simulator (2002)

    Google Scholar 

  19. Prinzel III, L.J., Pope, A.T., Palsson, O.S., Turner, M.J.: Method and apparatus for performance optimization through physical perturbation of task elements (2014)

    Google Scholar 

  20. Pope, A.T., Stephens, C.L., Jones, C.A.: Method and system for physiologically modulating action role-playing open world video games and simulations which use gesture and body image sensing control input devices. Google Patents (2015)

    Google Scholar 

  21. Pope, A.T., Stephens, C.L., Blanson, N.M.: Physiologically modulating videogames or simulations which use motion-sensing input devices (2014)

    Google Scholar 

  22. Sajnog, C.: How to Shoot Like a Navy Seal: Combat Marksmanship Fundamentals. Center Mass Group, LLC (2013)

    Google Scholar 

  23. Muñoz, J.E., Rubio, E., Cameirao, M., Bermúdez, S.: The biocybernetic loop engine: an integrated tool for creating physiologically adaptive videogames. In: 4th International Conference in Physiological Computing Systems, Madrid, España (2017)

    Google Scholar 

  24. Bontchev, B.: Adaptation in affective video games: a literature review. Cybern. Inf. Technol. 16, 3–34 (2016)

    MathSciNet  Google Scholar 

  25. Jacucci, G., Fairclough, S., Solovey, E.T.: Physiological computing. Computer 48, 12–16 (2015)

    Article  Google Scholar 

  26. Vourvopoulos, A., Faria, A.L., Cameirão, M.S., Bermudez i Badia, S.: RehabNet: a distributed architecture for motor and cognitive neuro-rehabilitation. In: 2013 IEEE 15th International Conference on e-Health Networking, Applications Services (Healthcom), pp. 454–459 (2013)

    Google Scholar 

  27. Fisher, J.G., Hatch, J.P., Rugh, J.D.: Biofeedback: Studies in Clinical Efficacy. Springer, Boston (2013)

    Google Scholar 

  28. Johnson, B.R.: Crucial Elements of Police Firearms Training. Looseleaf Law Publications (2007)

    Google Scholar 

  29. Muñoz, J.E., Gouveia, E.R., Cameirão, M.S., i Badia, S.B.: PhysioLab-a multivariate physiological computing toolbox for ECG, EMG and EDA signals: a case of study of cardiorespiratory fitness assessment in the elderly population. Multimed. Tools Appl., 1–26 (2017)

    Google Scholar 

  30. Plews, D.J., Scott, B., Altini, M., Wood, M., Kilding, A.E., Laursen, P.B.: Comparison of heart-rate-variability recording with smartphone photoplethysmography, Polar H7 chest strap, and electrocardiography. Int. J. Sport. Physiol. Perform. 12, 1324–1328 (2017)

    Article  Google Scholar 

  31. Sanei, S., Chambers, J.A.: EEG Signal Processing. Wiley, Hoboken (2013)

    Google Scholar 

  32. McMahan, T., Parberry, I., Parsons, T.D.: Evaluating electroencephalography engagement indices during video game Play. In: FDG (2015)

    Google Scholar 

  33. Kamzanova, A.T., Matthews, G., Kustubayeva, A.M., Jakupov, S.M.: EEG indices to time-on-task effects and to a workload manipulation (cueing). World Acad. Sci. Eng. Technol. 80, 19–22 (2011)

    Google Scholar 

  34. Fairclough, S., Gilleade, K.: Construction of the biocybernetic loop: a case study. In: Proceedings of the 14th ACM International Conference on Multimodal Interaction, p. 578. ACM, New York (2012)

    Google Scholar 

  35. Kennedy, R.S., Lane, N.E., Berbaum, K.S., Lilienthal, M.G.: Simulator sickness questionnaire: an enhanced method for quantifying simulator sickness. Int. J. Aviat. Psychol. 3, 203–220 (1993)

    Article  Google Scholar 

  36. Teasdale, N., Lavallière, M., Tremblay, M., Laurendeau, D., Simoneau, M.: Multiple exposition to a driving simulator reduces simulator symptoms for elderly drivers (2009)

    Google Scholar 

  37. Csikszentmihalyi, M.: Toward a psychology of optimal experience. Flow and the Foundations of Positive Psychology, pp. 209–226. Springer, Dordrecht (2014). https://doi.org/10.1007/978-94-017-9088-8_14

    Chapter  Google Scholar 

  38. Peifer, C.: Psychophysiological correlates of flow-experience. In: Engeser, S. (ed.) Advances in Flow Research, pp. 139–164. Springer, New York (2012). https://doi.org/10.1007/978-1-4614-2359-1_8

    Chapter  Google Scholar 

  39. Sandweiss, J.H.: Biofeedback and sports science. In: Sandweiss, J.H., Wolf, S.L. (eds.) Biofeedback and sports science, pp. 1–31. Springer, Boston (1985). https://doi.org/10.1007/978-1-4757-9465-6_1

    Chapter  Google Scholar 

  40. Grossman, D.: On Killing: The Psychological Cost of Learning to Kill in War and Society. Open Road Media, New York (2014)

    Google Scholar 

  41. Dey, A., Piumsomboon, T., Lee, Y., Billinghurst, M.: Effects of sharing physiological states of players in a collaborative virtual reality gameplay. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 4045–4056. ACM (2017)

    Google Scholar 

  42. Cacioppo, J.T., Tassinary, L.G., Berntson, G.: Handbook of Psychophysiology. Cambridge University Press, Cambridge (2007)

    Book  Google Scholar 

  43. Bombeke, K., et al.: Do not disturb: psychophysiological correlates of boredom, flow and frustration during VR gaming. In: Schmorrow, D.D., Fidopiastis, C.M. (eds.) AC 2018. LNCS (LNAI), vol. 10915, pp. 101–119. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91470-1_10

    Chapter  Google Scholar 

  44. Pugnetti, L., Meehan, M., Mendozzi, L.: Psychophysiological correlates of virtual reality: a review. Presence Teleoperators Virtual Environ. 10, 384–400 (2001)

    Article  Google Scholar 

  45. Wiederhold, B.K.: Virtual reality and applied psychophysiology. Appl. Psychophysiol. Biofeedback 30, 183–185 (2005)

    Article  Google Scholar 

  46. Stephens, C., et al.: Biocybernetic adaptation strategies: machine awareness of human engagement for improved operational performance. In: Schmorrow, Dylan D., Fidopiastis, Cali M. (eds.) AC 2018. LNCS (LNAI), vol. 10915, pp. 89–98. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-91470-1_9

    Chapter  Google Scholar 

  47. Fuchs, S., Schwarz, J.: Towards a dynamic selection and configuration of adaptation strategies in augmented cognition. In: Schmorrow, D.D., Fidopiastis, C.M. (eds.) AC 2017. LNCS (LNAI), vol. 10285, pp. 101–115. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-58625-0_7

    Chapter  Google Scholar 

  48. Pereira, M., Argelaguet, F., Millán, J. del R., Lécuyer, A.: Novice shooters with lower pre-shooting alpha power have better performance during competition in a virtual reality scenario. Front. Psychol. 9, 527 (2018)

    Google Scholar 

  49. Pugnetti, L., Mendozzi, L., Barberi, E., Rose, F.D., Attree, E.A.: Nervous system correlates of virtual reality experience. In: European Conference on Disability, Virtual Reality and Associated Technology (1996)

    Google Scholar 

  50. Friedman, D.: Brain-computer interfacing and virtual reality. In: Nakatsu, R., Rauterberg, M., Ciancarini, P. (eds.) Handbook of Digital Games and Entertainment Technologies, pp. 1–22. Springer, Singapore (2015). https://doi.org/10.1007/978-981-4560-52-8_2-1

    Chapter  Google Scholar 

  51. Malińska, M., Zużewicz, K., Bugajska, J., Grabowski, A.: Heart rate variability (HRV) during virtual reality immersion. Int. J. Occup. Saf. Ergon. 21, 47–54 (2015)

    Article  Google Scholar 

  52. Mason, P.H.: Recovering from the War: A Guide for All Veterans, Family Members. Friends and Therapists. Patience Press, High Springs (1998)

    Google Scholar 

  53. Muñoz, J.E., Rubio, E., Cameirao, M.S., Bermúdez i Badia, S.: Closing the loop in exergaming - health benefits of biocybernetic adaptation in senior adults. In: Proceedings of the 2018 Annual Symposium on Computer-Human Interaction in Play (2018)

    Google Scholar 

Download references

Acknowledgments

Authors would like to thank: (i) the J&F Alliance Group who financially supported the internship and VR development process, (ii) the National Institute of Aerospace (NIA) that supported the convergence of the stakeholders for this project, (iii) the PRISM team from NASA Langley for his very supportive feedback throughout the project development and (iv) Zeltech employees at Hampton facilities. Special thanks to personnel from the Hampton Police Department who participated actively in our studies; and Jeremy Sklute and Mike Priddy from J&F Alliance who helped in improving the system’s realism.

Author information

Authors and Affiliations

  1. System Design and Engineering Department, University of Waterloo, Waterloo, Canada

    John E. Muñoz

  2. NASA Langley Research Center, Hampton, USA

    Alan T. Pope

  3. Department of Computer and Systems Sciences at Stockholm University, Stockholm, Sweden

    Luis E. Velez

Authors
  1. John E. Muñoz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alan T. Pope
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luis E. Velez
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

JEM and ATP designed and defined the BioPhyS approach and the biocybernetic adaptation strategies. LEV developed the VR simulation and carried out the integration with the BL Engine. All authors revised and approved the current version of the manuscript.

Corresponding author

Correspondence to John E. Muñoz .

Editor information

Editors and Affiliations

  1. Medical University of Graz, Graz, Austria

    Andreas Holzinger

  2. NASA Langley Research Center, Hampton, VA, USA

    Alan Pope

  3. IT - Instituto de Telecomunicações, Lisbon, Portugal

    Hugo Plácido da Silva

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Muñoz, J.E., Pope, A.T., Velez, L.E. (2019). Integrating Biocybernetic Adaptation in Virtual Reality Training Concentration and Calmness in Target Shooting. In: Holzinger, A., Pope, A., Plácido da Silva, H. (eds) Physiological Computing Systems. PhyCS PhyCS PhyCS 2016 2017 2018. Lecture Notes in Computer Science(), vol 10057. Springer, Cham. https://doi.org/10.1007/978-3-030-27950-9_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-27950-9_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-27949-3

  • Online ISBN: 978-3-030-27950-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation