Augmenting Human Cognition with Adaptive Augmented Reality

  • Jayfus T. Doswell
  • Anna Skinner
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8534)


Wearable Augmented Reality (AR) combines research in AR, mobile/ubiquitous computing, and human ergonomics in which a video or optical see-through head mounted display (HMD) facilitates multi-modal delivery of contextually relevant and computer generated visual and auditory data over a physical, real-world environment. Wearable AR has the capability of delivering on-demand assistance and training across a variety of domains. A primary challenge presented by such advanced HCI technologies is the development of scientifically-grounded methods for identifying appropriate information presentation, user input, and feedback modalities in order to optimize performance and mitigate cognitive overload. A proposed framework and research methodology are described to support instantiation of physiologically-driven, adaptive AR to assess and contextually adapt to an individual’s environmental and cognitive state in real time. Additionally a use case within the medical domain is presented, and future research is discussed.


augmented cognition augmented reality context-awareness cognitive display mobile computing mixed-reality wearable computing 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Adeboye, O.O., Ochieng, E., Gillis, D., Doswell, J., Sofolahan, A., Dickens, C.: Context-Aware Augmented Reality System: A Medical First Responder Situational Awareness Interface. In: Poster Session. Medicine Meets Virtual Reality (MMVR) Conference. Newport Beach, California (2010)Google Scholar
  2. 2.
    Azimi, E., Doswell, J., Kazanzides, P.: Augmented reality goggles with an integrated tracking system for navigation in neurosurgery. In: Conference: Virtual Reality, IEEE Annual International Symposium, pp. 123–124 (2012)Google Scholar
  3. 3.
    Bellezza, F.S.: The Mind s Eye in Expert Memorizers Descriptions of Remembering. Metaphor and Symbolic Activity 7(3&4), 119–133 (1992)Google Scholar
  4. 4.
    Biocca, F., Owen, C., Tang, A., Bohil, C.: Attention issues in spatial information systems: directing mobile users’ visual attention using augmented reality. Journal of Management Information Systems 23(4), 163–184 (2007)CrossRefGoogle Scholar
  5. 5.
    Biocca, F., Tang, A., Owen, C., Xiao, F.: Attention funnel: omnidirectional 3D cursor for mobile augmented reality platforms. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1115–1122. ACM (April 2006)Google Scholar
  6. 6.
    Bonanni, L., Lee, C.H., Selker, T.: Attention-based design of augmented reality interfaces. In: CHI 2005 Extended Abstracts on Human Factors in Computing Systems, pp. 1228–1231. ACM (April 2005)Google Scholar
  7. 7.
    Carney, R.N., Levin, J.R.: Promoting Higher-Order Learning Benefits by Building Lower-Order Mnemonic Connections. Applied Cognitive Psychology 17, 563–575 (2003)CrossRefGoogle Scholar
  8. 8.
    Chai, L., et al.: Three-Dimensional Motion and Structure Estimation Using Inertial Sensors and Computer Vision for Augmented Reality. Presence 11(5), 474–492 (2002)CrossRefGoogle Scholar
  9. 9.
    Cox, B.D.: Children s Use of Mnemonic Strategies: Variability in Response to Metamemory Training. The Journal of Genetic Psychology 155(4), 423–442 (2001)CrossRefGoogle Scholar
  10. 10.
    Doswell, J., Sampson, O., Jeffrey, M.: WASA MIST Wearable Assistant for Real-Time Emergency Care Providers. In: 15th Annual International Meeting of the American Telemedicine Association (2010)Google Scholar
  11. 11.
    Doswell, J., Anderson, M., Wilson, K., Kazanzides, P., et al.: Wearable Augmented Reality for Medical First Response & Situational Awareness. In: Medicine Meets Virtual Reality (MMVR19) NextGen Conference. Newport Beach, California (2012)Google Scholar
  12. 12.
    Estrada, A., Keeley, J.A., Leduc, P.A., Bass, J.M., Rouse, T.N., Ramiccio, J.G., Rowe, T.L.: A novel approach in facilitating aviation emergency procedure learning and recall through an intuitive pictorial system. U.S. Army Aeromedical Research Laboratory Technical Report, No. 2007-07 (2007)Google Scholar
  13. 13.
    Fuchs, H., Livingston, M.A., Raskar, R., Keller, K., State, A., Crawford, J.R., Rademacher, P., Drake, S.H., Meyer, A.A.: Augmented reality visualization for laparoscopic surgery. In: Wells, W.M., Colchester, A.C.F., Delp, S.L. (eds.) MICCAI 1998. LNCS, vol. 1496, pp. 934–943. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  14. 14.
    Höllerer, T., Feiner, S.: Mobile Augmented Reality. In: Karimi, H., Hammad, A. (eds.) Telegeoinformatics: Location-Based Computing and Services, Taylor and Francis Books Ltd., London (2004)Google Scholar
  15. 15.
    Hollerer, T., Feiner, S., Hallawa, D., et al.: User Interface Management Techniques for Collaborative Mobile Augmented Reality. Computers and Graphics 25(5), 799–810 (2001), CrossRefGoogle Scholar
  16. 16.
    Hoff, W., Vincent, T.: Analysis of head pose accuracy in augmented reality. IEEE Trans. Visualization and Comp. Graphics 6 (2000)Google Scholar
  17. 17.
    Hughes, C., Stapleton, C.B., et al.: Mixed Reality in Education Entertainment, and Training. IEEE Computer Graphics and Applications, 24–30 (November/December 2005)Google Scholar
  18. 18.
    Kleinheksel, K.A., Summy, S.E.: Enhancing Student Learning and Social Behavior Through Mnemonic Strategies. TEACHING Exceptional Children 36(2), 30–35 (2003)Google Scholar
  19. 19.
    Kim, S., Dey, A.K.: Simulated augmented reality windshield display as a cognitive mapping aid for elder driver navigation. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 133–142. ACM (April 2009)Google Scholar
  20. 20.
    Learning and ID. Cognitive Load Theory,
  21. 21.
    Levin, M.E., Levin, J.R.: Scientific mnemonomies: Methods for maximizing more than memory. American Educational Research Journal 27, 301–321 (1990)CrossRefGoogle Scholar
  22. 22.
    Mastropieri, M.A., Scruggs, T.E.: Enhancing School Success with Mnemonic Strategies. Intervention in School & Clinic 33(4) (1998)Google Scholar
  23. 23.
    Medenica, Z., Kun, A.L., Paek, T., Palinko, O.: Augmented reality vs. street views: a driving simulator study comparing two emerging navigation aids. In: Proceedings of the 13th International Conference on Human Computer Interaction with Mobile Devices and Services, pp. 265–274. ACM (August 2011)Google Scholar
  24. 24.
    Miller, G.A.: The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychological Review 63, 81–97 (1956)CrossRefGoogle Scholar
  25. 25.
    Milgram, P., Kishino, A.F.: Taxonomy of Mixed RealityVisual Displays. IEICE Trans. Information and Systems E77-D(12), 1321–1329 (1994)Google Scholar
  26. 26.
    Nagao, K.: Agent augmented reality: Agents integrate the real world with cyberspace. In: Ishida, T. (ed.) Community Computing: Collaboration over Global Information Networks. John Wiley & Sons (1998)Google Scholar
  27. 27.
    Navarro, K.F.: Wearable, wireless brain computer interfaces in augmented reality environments. In: Proceedings of the International Conference on Information Technology: Coding and Computing, ITCC 2004, vol. 2, pp. 643–647. IEEE (October 2004)Google Scholar
  28. 28.
    Neumann, U., Majoros, A.: Cognitive, performance, and systems issues for augmented reality applications in manufacturing and maintenance. In: Proceedings of the Virtual Reality Annual International Symposium, pp. 4–11. IEEE (March 1998)Google Scholar
  29. 29.
    Raley, C., Stripling, R., Kruse, A., Schmorrow, D., Patrey, J.: Augmented cognition overview: improving information intake under stress. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 48(10), pp. 1150–1154. SAGE Publications (2004)Google Scholar
  30. 30.
    Sadda, P., Azimi, E., Jallo, G., Kazinzides, P.: Surgical Navigation with a Head-Mounted Tracking System and Display. In: Medicine Meets Virtual Reality (MMVR20) NextGen Conference, San Diego, California (2013)Google Scholar
  31. 31.
    Sauer, F., Wenzel, F., Vogt, S., Tao, Y., Genc, Y., Bani-Hashemi, A., et al.: Augmented Workspace: designing an AR testbed. In: Proc. IEEE Intl. Symp. on Augmented Reality (ISAR) (2000)Google Scholar
  32. 32.
    Schmorrow, D.D., Kruse, A.A.: DARPA’s Augmented Cognition Program-tomorrow’s human computer interaction from vision to reality: building cognitively aware computational systems. In: Proceedings of the 2002 IEEE 7th Conference on Human Factors and Power Plants, pp. 7–1. IEEE (2002)Google Scholar
  33. 33.
    Scherer, R., Chung, M., Lyon, J., Cheung, W., Rao, R.P.: Interaction With Virtual And Augmented Reality Environments Using Non-Invasive Brain-Computer Interfacing. In: 1st International Conference on Applied Bionics and Biomechanics (October 2010)Google Scholar
  34. 34.
    Sweller, J.: Implications of cognitive load theory for multimedia learning. In: Mayer, R.E. (ed.) The Cambridge Handbook of Multimedia Learning, pp. 19–30. Cambridge University Press, Cambridge (2005)CrossRefGoogle Scholar
  35. 35.
    Tang, A., Owen, C., Biocca, F., Mou, W.: Comparative effectiveness of augmented reality in object assembly. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 73–80. ACM (April 2003)Google Scholar
  36. 36.
    Tang, A., Owen, C.B., Biocca, F., Mou, W.: Experimental Evaluation of Augmented Reality in Object Assembly Task. In: ISMAR, vol. 2, p. 265 (2002)Google Scholar
  37. 37.
    Van Krevelen, D.W.F., Poelman, R.: A survey of augmented reality technologies, applications and limitations. International Journal of Virtual Reality 9(2), 1 (2010)Google Scholar
  38. 38.
    Wilson, J., Doswell, J.: Using Wearable Augmented Reality as a Clinical-Decision Support Tool. mHealth Summit. Session on Emergency Response & Mobile Technology, Washington, DC (2010)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Jayfus T. Doswell
    • 1
  • Anna Skinner
    • 2
  1. 1.JuxtopiaBaltimoreUSA
  2. 2.AnthroTronixSilver SpringUSA

Personalised recommendations