An Eye-Tracking Study of Integrative Spatial Cognition over Diagrammatic Representations

  • Atsushi Shimojima
  • Yasuhiro Katagiri
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6222)


Spatial representations, such as maps, charts, and graphs, convey different levels of information, depending on how their elements are grouped into different units of objects. Therefore, how people set boundaries to graphical objects to be interpreted and how they maintain the object boundaries during the given task are two important problems in understanding the way people utilize spatial representations for problem-solving. Table comprehension process was experimentally investigated in terms of eye gaze control behaviors when people were required to read off information distributed over large-scale objects, e.g., a row or a column, of the given table. Evidence was found that a large-scale object can be bounded by a single attentional shift to it, and that they can be retained as coherent objects for subsequent reference. These findings suggest the existence of a higher-order information processing in the comprehension of a spatial representation, based on rather intricate processes of attention management.


spatial representation object-based attention visual index  eye-tracking embodied cognition situated cognition 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ballard, D.H., Hayhoe, M.M., Pook, P.K., Rao, R.P.N.: Deictic codes for the embodiment of cognition. Behavioral and Brain Sciences 20(4), 723–767 (2001)Google Scholar
  2. 2.
    Barwise, J., Etchemendy, J.: Visual information and valid reasoning. In: Allwein, G., Barwise, J. (eds.) Logical Reasoning with Diagrams, pp. 3–25. Oxford University Press, Oxford (1990)Google Scholar
  3. 3.
    Bertin, J.: Semiology of Graphics: Diagrams, Networks, Maps. The University of Wisconsin Press (1973)Google Scholar
  4. 4.
    Bertin, J.: Graphics and Graphic Information. Walter de Gruyter, Berlin (1981) (Originally published in France in 1977)Google Scholar
  5. 5.
    Blignaut, P.: Fixation identification: The optimum threshold for a dispersion algorithm. Attention, Perception, & Psychophysics 71(4), 881–895 (2009)CrossRefGoogle Scholar
  6. 6.
    Driver, J., Baylis, G.C.: Attention and visual object segmentation. In: Driver, J., Baylis, G.C., Parasuraman, R. (eds.) The Attentive Brain, pp. 299–325. MIT Press, Cambridge (1998)Google Scholar
  7. 7.
    Duncan, J.: Selective attention and the organization of visual information. Journal of Experimental Psychology: General 113(4), 501–517 (1984)CrossRefGoogle Scholar
  8. 8.
    Gilhooly, K.J., Wood, M., Kinnear, P.R., Green, C.: Skill in map reading and memory for maps. Quarterly Journal of Experimental Psychology 40A, 87–107 (1988)Google Scholar
  9. 9.
    Grawemeyer, B., Cox, R.: The effects of users’ background diagram knowledge and task characteristics upon information display selection. In: Stapleton, G., Howse, J., Lee, J. (eds.) Diagrams 2008. LNCS (LNAI), vol. 5223, pp. 321–334. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  10. 10.
    Guthrie, J.T., Weber, S., Kimmerly, N.: Searching documents: Cognitive processes and deficits in understanding graphs, tables, and illustrations. Contemporary Educational Psychology 18, 186–221 (1993)CrossRefGoogle Scholar
  11. 11.
    Kahneman, D., Treisman, A., Gibbs, B.J.: The reviewing of object files: Object-speci?c integration of information. Cognitive Psychology 24(2), 175–219 (1992)CrossRefGoogle Scholar
  12. 12.
    Kosslyn, S.M.: Elements of Graph Design. W. H. Freeman and Company, New York (1994)Google Scholar
  13. 13.
    Kramer, A.F., Jacobson, A.: Perceptual organization and focused attention: the role of objects and proximity in visual processing. Perception & Psychophysics 50(3), 267–284 (1991)Google Scholar
  14. 14.
    Lamme, V.A.F., Rodriguez-Rodriguez, V., Spekreojse, H.: Separate processing dynamics for texture elements, bundaries and surfaces in primary visual cortex of the macaque monkey. Cerebral Cortex 9, 406–413 (1999)CrossRefGoogle Scholar
  15. 15.
    Lowe, R.K.: Diagram prediction and higher order structures in mental representation. Research in Science Education 24, 208–216 (1994)CrossRefGoogle Scholar
  16. 16.
    O’Craven, K.M., Downing, P.E., Kanwisher, N.K.: fMRI evidence for objects as the units of attentional selection. Nature 401, 584–587 (1999)CrossRefGoogle Scholar
  17. 17.
    Olivier, P.: Diagrammatic reasoning: an artificial intelligence perspective. In: Blackwell, A.F. (ed.) Thinking with Diagrams, pp. 63–78. Kluwer Academic Publishers, Dordrecht (2001)Google Scholar
  18. 18.
    Pinker, S.: A theory of graph comprehension. In: Aritificial Intelligence and the Future of Testing, pp. 73–126. L. Erlbaum Associates, Mahwah (1990)Google Scholar
  19. 19.
    Pylyshyn, Z.: The role of location indexes in spatial perception: A sketch of the finst spatial-index model. Cognition 32, 65–97 (1989)CrossRefGoogle Scholar
  20. 20.
    Pylyshyn, Z.: Seeing and Visualizing: It’s Not What You Think. The MIT Press, Cambridge (2003)Google Scholar
  21. 21.
    Ratwani, R.M., Trafton, J.G., Boehm-Davis, D.A.: Thinking graphically: Connecting vision and cognition during graph comprehension. Journal of Experimental Psychology: Applied 14(1), 36–49 (2008)CrossRefGoogle Scholar
  22. 22.
    Roelfsema, P.R.: Cortical algorithms for perceptual grouping. Annual Review of Neuroscience 29, 203–227 (2006)CrossRefGoogle Scholar
  23. 23.
    Roelfsema, P.R., Lamme, V.A.F., Spekreijse, H.: The implementation of visual routines. Vision Research 40, 1385–1411 (2000)CrossRefGoogle Scholar
  24. 24.
    Roelfsema, P.R., Lamme, V.A.F., Spekreojse, H.: Object-based attention in the primary visual cortex of the macaque monkey. Nature 395, 376–381 (1998)CrossRefGoogle Scholar
  25. 25.
    Shimojima, A.: Operational constraints in diagrammatic reasoning. In: Barwise, J., Allwein, G. (eds.) Logical Reasoning with Diagrams, pp. 27–48. Oxford University Press, Oxford (1995)Google Scholar
  26. 26.
    Shimojima, A.: Derivative meaning in graphical representations. In: Proceedings of the 1999 IEEE Symposium on Visual Languages, pp. 212–219. IEEE Computer Society, Los Alamitos (1999)CrossRefGoogle Scholar
  27. 27.
    Shimojima, A., Katagiri, Y.: An eye-tracking study of exploitations of spatial constraints in diagrammatic reasoning. In: Stapleton, G., Howse, J., Lee, J. (eds.) Diagrams 2008. LNCS (LNAI), vol. 5223, pp. 74–88. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  28. 28.
    Shimojima, A., Katagiri, Y.: Hypothetical drawing in embodied spatial reasoning. In: Proceedings of the 30th Annual Meeting of the Cognitive Science Society, pp. 2247–2252 (2008)Google Scholar
  29. 29.
    Spivey, M.J., Richardson, D.C., Fitneva, S.A.: Thinking outside the brain: Spatial indices to visual and linguistic information. In: Henderson, J.M., Ferreira, F. (eds.) The Interface of Language, Vision, and Action: Eye Movements and the Visual World, pp. 161–189. Psychology Press, San Diego (2004)Google Scholar
  30. 30.
    Stenning, K., Oberlander, J.: A cognitive theory of graphical and linguistic reasoning: Logic and implementation. Cognitive Science 19(1), 97–140 (1995)CrossRefGoogle Scholar
  31. 31.
    Ullman, S.: Visual routines. Cognition 18, 97–159 (1984)CrossRefGoogle Scholar
  32. 32.
    Wainer, H.: Understanding graphs and tables. Educaional Researcher 21, 14–23 (1992)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Atsushi Shimojima
    • 1
  • Yasuhiro Katagiri
    • 2
  1. 1.Doshisha UniversityKyotoJapan
  2. 2.Future University HakodateHokkaidoJapan

Personalised recommendations