Human Performance with Complex Technology: How Visual Cognition Is Critical to Enhanced Performance with Aided Target Recognition (AiTR)

  • Gabriella Brick LarkinEmail author
  • Michael N. Geuss
  • Alfred Yu
  • Chloe Callahan-Flintoft
  • Joe Rexwinkle
  • Chou P. Hung
  • Brent J. Lance
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1201)


Technology advances in artificial intelligence (AI) and augmented or mixed reality (AR) offer the potential for advanced performance capabilities in the military and commercial sectors. However, the impact of utilizing such technology on users’ normative perceptual, attentional, and higher-level cognitive processes is not well understood. To take full advantage of current and future technological advancements, systems designers for the Army must better understand how human visual cognition changes in the face of the novel visual stimuli provided by these technologies. Here, we present an approach anchored in foundational cognitive research to derive principles for how humans understand, interact with, and are cognitively altered by the interactions between AI and AR. We will discuss our approach in the context of a specific application, Aided Target Recognition (AiTR). We will discuss a series of planned research efforts, the foundational findings supporting these efforts, and their potential implications for AiTR development.


Applied perception Augmented reality Visual cognition AiTR 


  1. 1.
    Larkin, G.B., Geuss, M., Yu, A., Rexwinkle, J., Callahan-Flintoft, C., Bakdash, J., Swoboda, J., Lieberman, G., Hung, C.P., Moore, S., Lance, B.J.: Augmented target recognition display recommendations. J. Defense Syst. Inf. Anal. Center Winter 2020. (in press)Google Scholar
  2. 2.
    Haberman, J., Whitney, D.: Ensemble perception: summarizing the scene and broadening the limits of visual processing. From Perception to Consciousness: Searching with Anne Treisman, pp. 339–349 (2012)Google Scholar
  3. 3.
    Balas, B., Nakano, L., Rosenholtz, R.: A summary-statistic representation in peripheral vision explains visual crowding. J. Vis. 9(12), 13 (2009)CrossRefGoogle Scholar
  4. 4.
    Alvarez, G.A., Oliva, A.: Spatial ensemble statistics are efficient codes that can be represented with reduced attention. Proc. Natl. Acad. Sci. 106(18), 7345–7350 (2009)CrossRefGoogle Scholar
  5. 5.
    Geisler, W.S.: Visual perception and the statistical properties of natural scenes. Ann. Rev. Psychol. 59, 167–192 (2008)CrossRefGoogle Scholar
  6. 6.
    Groen, I.I., Silson, E.H., Baker, C.I.: Contributions of low-and high-level properties to neural processing of visual scenes in the human brain. Phil. Trans. Roy. Soc. B Biol. Sci. 372(1714), 20160102 (2017)CrossRefGoogle Scholar
  7. 7.
    Geisler, W.S., Diehl, R.L.: Bayesian natural selection and the evolution of perceptual systems. Phil. Trans. Roy. Soc. Lond. Ser. B Biol. Sci. 357(1420), 419–448 (2002)CrossRefGoogle Scholar
  8. 8.
    Hung, C.P., Cui, D., Chen, Y.P., Lin, C.P., Levine, M.R.: Correlated activity supports efficient cortical processing. Front. Comput. Neurosci. 8, 171 (2015)Google Scholar
  9. 9.
    Haberman, J., Whitney, D.: Rapid extraction of mean emotion and gender from sets of faces. Curr. Biol. 17(17), R751–R753 (2007)CrossRefGoogle Scholar
  10. 10.
    Haberman, J., Whitney, D.: Seeing the mean: ensemble coding for sets of faces. J. Exp. Psychol. Hum. Percept. Perform. 35(3), 718 (2009)CrossRefGoogle Scholar
  11. 11.
    Baddeley, R., Abbott, L.F., Booth, M.C., Sengpiel, F., Freeman, T., Wakeman, E.A., Rolls, E.T.: Responses of neurons in primary and inferior temporal visual cortices to natural scenes. Proc. Roy. Soc. Lond. Ser. B Biol. Sci. 264(1389), 1775–1783 (1997)CrossRefGoogle Scholar
  12. 12.
    De Cesarei, A., Loftus, G.R., Mastria, S., Codispoti, M.: Understanding natural scenes: contributions of image statistics. Neurosci. Biobehav. Rev. 74, 44–57 (2017)CrossRefGoogle Scholar
  13. 13.
    Dima, D.C., Perry, G., Singh, K.D.: Spatial frequency supports the emergence of categorical representations in visual cortex during natural scene perception. NeuroImage 179, 102–116 (2018)CrossRefGoogle Scholar
  14. 14.
    Gaspelin, N., Leonard, C.J., Luck, S.J.: Direct evidence for active suppression of salient-but-irrelevant sensory inputs. Psychol. Sci. 26(11), 1740–1750 (2015)CrossRefGoogle Scholar
  15. 15.
    Geuss, M.N., Larkin, G., Swoboda, J., Yu, A., Bakdash, J., White, T., Lance, B.: Intelligent squad weapon: challenges to displaying and interacting with artificial intelligence in small arms weapon systems. In: Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications, vol. 11006, p. 110060 V (2019). International Society for Optics and PhotonicsGoogle Scholar
  16. 16.
    Gaspelin, N., Luck, S.J.: The role of inhibition in avoiding distraction by salient stimuli. Trends Cogn. Sci. 22(1), 79–92 (2018)CrossRefGoogle Scholar
  17. 17.
    Wyble, B., Callahan-Flintoft, C., Chen, H., Marinov, T., Sarkar, A., Bowman, H.: Understanding visual attention with RAGNAROC: a reflexive attention gradient through neural AttRactOr Competition. bioRxiv 406124 (2018)Google Scholar
  18. 18.
    Müller, M.M., Gundlach, C., Forschack, N., Brummerloh, B.: It takes two to tango: Suppression of task-irrelevant features requires (spatial) competition. NeuroImage 178, 485 (2018)CrossRefGoogle Scholar
  19. 19.
    Chen, J.Y.: Concurrent performance of military tasks and robotics tasks: effects of automation unreliability and individual differences. In: Proceedings of the 4th ACM/IEEE International Conference on Human Robot Interaction, pp. 181–188. ACM (209)Google Scholar
  20. 20.
    Friedman, J.A., Baker, J.D., Mellers, B.A., Tetlock, P.E., Zeckhauser, R.: The value of precision in probability assessment: evidence from a large-scale geopolitical forecasting tournament. Int. Stud. Q. 62(2), 410–422 (2018)Google Scholar
  21. 21.
    Gonzalez, R., Wu, G.: On the shape of the probability weighting function. Cogn. Psychol. 38(1), 129–166 (1999)CrossRefGoogle Scholar
  22. 22.
    Lipkus, I.M., Samsa, G., Rimer, B.K.: General performance on a numeracy scale among highly educated samples. Med. Decis. Mak. 21(1), 37–44 (2001)CrossRefGoogle Scholar
  23. 23.
    Brunyé, T.T., Gardony, A.L.: Eye tracking measures of uncertainty during perceptual decision making. Int. J. Psychophysiol. 120, 60–68 (2017)CrossRefGoogle Scholar
  24. 24.
    Kietzmann, T.C., König, P.: Effects of contextual information and stimulus ambiguity on overt visual sampling behavior. Vision. Res. 110(Part A), 76–86 (2015)CrossRefGoogle Scholar
  25. 25.
    Nuthmann, A.: Fixation durations in scene viewing: modeling the effects of local image features, oculomotor parameters, and task. Psycho. Bull. Rev. 24(2), 370–392 (2017)CrossRefGoogle Scholar
  26. 26.
    Johansson, G.: Visual perception of biological motion and a model for its analysis. Percept. Psychophys. 14(2), 201–211 (1973)CrossRefGoogle Scholar
  27. 27.
    Owens, D.A., Antonoff, R.J., Francis, E.L.: Biological motion and nighttime pedestrian conspicuity. Hum. Factors 36(4), 718–732 (1994)CrossRefGoogle Scholar
  28. 28.
    Yu, A.B., Zacks, J.M.: How are bodies special? effects of body features on spatial reasoning. Q. J. Exp. Psychol. 69(6), 1210–1226 (2015)CrossRefGoogle Scholar
  29. 29.
    Insafutdinov, E., Pishchulin, L., Andres, B., Andriluka, M., Schiele, B.: Deepercut: adeeper, stronger, and faster multi-person pose estimation model. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) Computer Vision – ECCV 2016, Lecture Notes in Computer Science, vol 9910. Springer, Cham (2016)Google Scholar
  30. 30.
    Cao, Z., Hidalgo, G., Simon, T., Wei, S.-E., Sheikh, Y.: OpenPose: realtime multi-person 2D pose estimation using part affinity fields. IEEE Trans. Pattern Anal. Mach. Intell. (2019)Google Scholar

Copyright information

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2021

Authors and Affiliations

  • Gabriella Brick Larkin
    • 1
    Email author
  • Michael N. Geuss
    • 1
  • Alfred Yu
    • 1
  • Chloe Callahan-Flintoft
    • 1
  • Joe Rexwinkle
    • 1
  • Chou P. Hung
    • 1
  • Brent J. Lance
    • 1
  1. 1.Combat Capabilities Development Command Army Research Laboratory, Human Research and Engineering DirectorateAberdeen Proving GroundUSA

Personalised recommendations