Bio-reckoning: Perceptual User Interface Design for Military Training

  • Tami Griffith
  • Deanna Rumble
  • Pankaj Mahajan
  • Cali M. Fidopiastis
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8027)


Simulation based training is one way to attain operational realism for training complex military tasks in a safe, task relevant manner. For successful transfer of knowledge, skills, and abilities to the dynamically changing military environment, the human-computer interface should minimally support learning during the training process and provide congruent action plans that facilitate understanding of the overall training goal. While there are emerging controller technologies, simulators still rely on such input devices as mouse and keyboard. These devices potentially cause information and training bottlenecks as they limit naturalistic interactivity within the more advanced serious gaming platforms. Given the shortcomings of current interface design, we suggest a human-computer interface framework that includes perceptual user interface components and an open source serious game testbed. We discuss a multimodal framework called bio-reckoning that integrates brain-computer interface techniques, eye tracking, and facial recognition within EDGE, the U.S. Army’s newest serious game based training tool.


simulation based training perceptual user interfaces braincomputer interfaces serious games military training augmented cognition 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Smith, R.: The long history of gaming in military training. Simulat Gaming 41(1), 6–19 (2010)CrossRefGoogle Scholar
  2. 2.
    Grant, S.T., Barnett, J.S.: Evaluation of wearable simulation interface for military training. Hum. Fact. (2012), doi:10.1177/0018720812466892Google Scholar
  3. 3.
    Rahman, M., Balakrishanan, G., Bergin, T.: Designing human–machine interfaces for naturalistic perceptions,decisions and actions occurring in emergency situations. Theoretical Issues in Ergonomics Science 13(3), 358–379 (2012)CrossRefGoogle Scholar
  4. 4.
    Smith, R.: Game Impact Theory: The Five Forces That Are Driving the Adoption of Game Technologies within Multiple Established Industries. Games and Society Yearbook, 1–32 (2006)Google Scholar
  5. 5.
    Smith, R.: The Disruptive Potential of Game Technologies. Research Technology Management 50(2), 57–64 (2007)Google Scholar
  6. 6.
    Klochek, C., MacKenzie, I.S.: Performance measures of game controllers in a three-dimensional environment. In: Proceedings of Graphics Interface 2006, pp. 73–79. CIPS, Toronto (2006)Google Scholar
  7. 7.
    Fischer, L., Oliveira, G., Osmari, D., Nedel, L.: Finding Hidden Objects in Large 3D Environments: the Supermarket Problem. In: Proceedings of 2011 XIII Symposium on Virtual Reality, pp. 79–88. IEEE Press, Brazil (2011)CrossRefGoogle Scholar
  8. 8.
    Thorpe, A., Ma, M., Oikonomou, A.: History and Alternative Game Input Methods. In: The proceedings of the 2011 16th International Conference on Computer Games (CGAMES), Derby, UK, pp. 76–93 (2011)Google Scholar
  9. 9.
    Xia, L., Chen, C.C., Aggerwal, J.K.: Human Detection using depth information by Kinect. In: The proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 15–22. IEEE (2011)Google Scholar
  10. 10.
  11. 11.
  12. 12.
  13. 13.
    Mohanram, N.K.: A Speech-based Quiz Game. Final Dissertation Report (2003) Google Scholar
  14. 14.
    Sadun, E., Sande, S.: Talking to Siri: Learning the Language of Apple’s Intelligent Assistant. Que Publishing, USA (2012)Google Scholar
  15. 15.
    Turk, M., Robertson, G.: Perceptual user interfaces. Communications of the ACM 43(3), 33–34 (2000)CrossRefGoogle Scholar
  16. 16.
    Nicholson, D.M., Fidopiastis, C.M., Davis, L.D., Schmorrow, D.D., Stanney, K.M.: An adaptive instructional architecture for training and education. In: Schmorrow, D.D., Reeves, L.M. (eds.) Augmented Cognition, HCII 2007. LNCS (LNAI), vol. 4565, pp. 380–384. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  17. 17.
    Turk, M.: Perceptual User Interfaces. In: NSF Workshop (2006)Google Scholar
  18. 18.
    Cohen, P.R., Johnston, M., McGee, D., Oviatt, S., Pittman, J., Smith, I., Chen, L., Clow, J.: QuickSet: Multimodal Interaction for Simulation Set-up and Control. In: The proceedings of the Fifth Conference on Applied Natural Language Processing, Washington, DC, USA (March 1997)Google Scholar
  19. 19.
    Pittman, J., Smith, I., Cohen, P.R., Oviatt, S.L., Yang, T.C.: QuickSet: A multimodal interface for militarysimulation. In: The Proceedings of the Sixth Conference on Computer Generated Forces and Behavioral Representation, pp. 217–224. Univ. of Central Florida, Orlando (1996)Google Scholar
  20. 20.
    Fidopiastis, C.M., Wiederhold, M.: Mindscape Retuning and Brain Reorganization with Hybrid Universes: The Future of Virtual Rehabilitation. In: Schmorrow, D., Cohn, J., Nicholson, D. (eds.) The PSI Handbook of Virtual Environments for Training & Education: Developments for the Military and Beyond, vol. 3, pp. 427–434. Praeger Security International, Westport (2008)Google Scholar
  21. 21.
    Lane, S.H., Marshall, H., Roberts, T.: Control interface for driving interactive characters in immersive virtual environments Technical Report, US Army Research (2006)Google Scholar
  22. 22.
    Lalor, E.C., Kelly, S.P., Finucane, C., Burke, R., Smith, R., Reilly, R.B., McDarby, G.: Steady-state VEP-based brain-computer interface control in an immersive 3D gaming environment. Eurasip J. on Appl. Sign Process. 19, 3156–3164 (2005)CrossRefGoogle Scholar
  23. 23.
    Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtschellere, G., Vaughan, T.M.: Brain-computer interfaces for communication and control. Clin. Neurophysiol. 113(6), 767–791 (2002)CrossRefGoogle Scholar
  24. 24.
    Smith, J.D., Graham, T.C.N.: Use of eye movements for video game control. In: The proceedings of the ACE 2006, Hollywood, California, USA, June 14-16 (2006)Google Scholar
  25. 25.
    Hayhoe, M., Ballard, D.: Eye movements in natural behavior. Trends Cogn. Sci. 9(4), 188–194 (2005)CrossRefGoogle Scholar
  26. 26.
    Tanaka, J., Gautghier, I.: Expertise in object and face recognition. The Psychology of Learning and Motivation 36, 83–125 (1997)CrossRefGoogle Scholar
  27. 27.
    Darken, R., McDowell, P., Murphy, C.: Open Source Game Engines: Disruptive Technologies in Training and Education. In: Proceedings of theInterservice/Industry Training, Simulation and Education Conference (I/ITSEC). National Defense Industrial Association, Orlando (2005)Google Scholar
  28. 28.
    Dwyer, T., Griffith, T., Maxwell, D.: Rapid Simulation Development Using a Game Engine-Enhanced Dynamic Geo-Social Environment. In: The proceedings of Interservice/Industry Training, Simulation & Education Conference (I/ITSEC). NTSA, Orlando (2011)Google Scholar
  29. 29.
    Eisenberger, R., Jones, J.R., Stinglhamber, F., Shanock, L., Randall, A.T.: Flow Experiences at work; for high need achievers alone? J. Organ Behav. 26, 755–775 (2005)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Tami Griffith
    • 1
  • Deanna Rumble
    • 2
  • Pankaj Mahajan
    • 2
  • Cali M. Fidopiastis
    • 2
  1. 1.Human Research and Engineering Directorate, Simulation and Training Technology CenterU.S. Army Research LaboratoryOrlandoUSA
  2. 2.School of Health ProfessionsUniversity of Alabama at BirminghamBirminghamUSA

Personalised recommendations