Studying Biases in Visualization Research: Framework and Methods

  • André Calero ValdezEmail author
  • Martina Ziefle
  • Michael Sedlmair


In this chapter, we propose and discuss a lightweight framework to help organize research questions that arise around biases in visualization and visual analysis. We contrast our framework against the cognitive bias codex by Buster Benson. The framework is inspired by Norman’s Human Action Cycle and classifies biases into three levels: perceptual biases, action biases, and social biases. For each of the levels of cognitive processing, we discuss examples of biases from the cognitive science literature and speculate how they might also be important to the area of visualization. In addition, we put forward a methodological discussion on how biases might be studied on all three levels, and which pitfalls and threats to validity exist. We hope that the framework will help spark new ideas and guide researchers that study the important topic of biases in visualization.



The authors wish to thank the reviewers. This work was partly funded by the German Research Council DFG excellence cluster “Integrative Production Technology in High Wage Countries”, and the FFG project 845898 (VALID).


  1. 1.
    Ahissar E, Assa E (2016) Perception as a closed-loop convergence process. Elife 5(e12):830Google Scholar
  2. 2.
    Baird JC, Noma EJ (1978) Fundamentals of scaling and psychophysics. Wiley, New YorkGoogle Scholar
  3. 3.
    Beecham R, Dykes J, Meulemans W, Slingsby A, Turkay C, Wood J (2017) Map lineups: effects of spatial structure on graphical inference. IEEE Trans Visual Comput Graphics 23(1):391–400CrossRefGoogle Scholar
  4. 4.
    Birch SA, Bloom P (2007) The curse of knowledge in reasoning about false beliefs. Psychol Sci 18(5):382–386CrossRefGoogle Scholar
  5. 5.
    Brath R (2012) Multi-attribute glyphs on Venn and Euler diagrams to represent data and aid visual decoding. In: 3rd international workshop on euler diagrams, p 122Google Scholar
  6. 6.
    Calero Valdez A, Ziefle M, Sedlmair M (2017) A framework for studying biases in visualization research. In: Proceedings of the 2nd DECISIVe workshop 2017 held at IEEE VISGoogle Scholar
  7. 7.
    Calero Valdez A, Ziefle M, Sedlmair M (2018) Priming and anchoring effects in visualization. IEEE Trans Visual Comput Graphics 24(1):584–594CrossRefGoogle Scholar
  8. 8.
    Cornsweet TN (1962) The staircase-method in psychophysics. Am J Psychol 75(3):485–491CrossRefGoogle Scholar
  9. 9.
    Cumming G (2012) Understanding the new statistics: effect sizes, confidence intervals, and meta-analysis. Routledge, LondonGoogle Scholar
  10. 10.
    Dimara E, Dragicevic P, Bezerianos A (2016) Accounting for availability biases in information visualization. arXiv preprint arXiv:161002857
  11. 11.
    Dragicevic P, Jansen Y (2014) Visualization-mediated alleviation of the planning fallacy. In: IEEE VIS 2014Google Scholar
  12. 12.
    Ellis G, Dix A (2015) Decision making under uncertainty in visualisation? In: IEEE VIS workshop on visualization for decision making under uncertainty (VDMU)Google Scholar
  13. 13.
    Gilbert DT, Brown RP, Pinel EC, Wilson TD (2000) The illusion of external agency. J Pers Soc Psychol 79(5):690CrossRefGoogle Scholar
  14. 14.
    Hamilton DL, Gifford RK (1976) Illusory correlation in interpersonal perception: a cognitive basis of stereotypic judgments. J Exp Soc Psychol 12(4):392–407CrossRefGoogle Scholar
  15. 15.
    Harrison L, Yang F, Franconeri S, Chang R (2014) Ranking visualizations of correlation using Weber’s law. In: Proceedings of the ieee information visualization symposium (InfoVis), vol 20(12), pp 1943–1952CrossRefGoogle Scholar
  16. 16.
    Hecht S (1924) The visual discrimination of intensity and the Weber-Fechner law. J Gen Physiol 7(2):235–267CrossRefGoogle Scholar
  17. 17.
    Johnston JC, McClelland JL (1973) Visual factors in word perception. Attention Percept Psychophys 14(2):365–370CrossRefGoogle Scholar
  18. 18.
    Kahneman D (2012) A proposal to deal with questions about priming effects. Nature 490Google Scholar
  19. 19.
    Karlsson N, Loewenstein G, Seppi D (2009) The ostrich effect: selective attention to information. J Risk Uncertainty 38(2):95–115CrossRefGoogle Scholar
  20. 20.
    Kay M, Heer J (2016) Beyond Weber’s law: a second look at ranking visualizations of correlation. IEEE Trans Visual Comput Graphics 22(1):469–478CrossRefGoogle Scholar
  21. 21.
    Lerner MJ (1980) The belief in a just world. In: The Belief in a just World. Springer, Berlin, pp 9–30CrossRefGoogle Scholar
  22. 22.
    Levitt H (1971) Transformed up-down methods in psychoacoustics. J Acoust Soc Am 49(2B):467–477CrossRefGoogle Scholar
  23. 23.
    LoBue V (2010) And along came a spider: an attentional bias for the detection of spiders in young children and adults. J Exp Child Psychol 107(1):59–66CrossRefGoogle Scholar
  24. 24.
    Meyer DE, Schvaneveldt RW (1971) Facilitation in recognizing pairs of words: evidence of a dependence between retrieval operations. J Exp Psychol 90(2):227CrossRefGoogle Scholar
  25. 25.
    Morgan M, Hole GJ, Glennerster A (1990) Biases and sensitivities in geometrical illusions. Vision Res 30(11):1793–1810CrossRefGoogle Scholar
  26. 26.
    Norman D (1988) The design of everyday things. Doubled CurrencyGoogle Scholar
  27. 27.
    Park B, Rothbart M (1992) Perception of out-group homogeneity and levels of social categorization: memory for the subordinate attributes of in-group and out-group members. J Pers Soc Psychol 42(6):1051CrossRefGoogle Scholar
  28. 28.
    Pronin E, Lin DY, Ross L (2002) The bias blind spot: perceptions of bias in self versus others. Pers Soc Psychol Bull 28(3):369–381CrossRefGoogle Scholar
  29. 29.
    Rice ML, Hadley PA, Alexander AL (1993) Social biases toward children with speech and language impairments: a correlative causal model of language limitations. Appl Psycholinguistics 14(4):445–471CrossRefGoogle Scholar
  30. 30.
    Schooler JW (2014) Metascience could rescue the ‘replication crisis’. Nature 515(7525):9CrossRefGoogle Scholar
  31. 31.
    Sedlmair M, Meyer M, Munzner T (2012) Design study methodology: reflections from the trenches and the stacks. IEEE Trans Visual Comput Graphics 18(12):2431–2440CrossRefGoogle Scholar
  32. 32.
    Seizova-Cajic T, Gillam B (2006) Biases in judgments of separation and orientation of elements belonging to different clusters. Vision Res 46(16):2525–2534CrossRefGoogle Scholar
  33. 33.
    Strack F, Mussweiler T (1997) Explaining the enigmatic anchoring effect: mechanisms of selective accessibility. J Pers Soc Psychol 73(3):437CrossRefGoogle Scholar
  34. 34.
    Tufte E, Graves-Morris P (2014) The visual display of quantitative information (original publish 1983)Google Scholar
  35. 35.
    Tversky A, Kahneman D (1973) Availability: a heuristic for judging frequency and probability. Cogn Psychol 5(2):207–232CrossRefGoogle Scholar
  36. 36.
    Verbeiren T, Sakai R, Aerts J (2014) A pragmatic approach to biases in visual data analysis. In: IEEE VIS 2014Google Scholar
  37. 37.
    Wickham H, Cook D, Hofmann H, Buja A (2010) Graphical inference for infovis. IEEE Trans Visual Comput Graphics 16(6):973–979CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  • André Calero Valdez
    • 1
    Email author
  • Martina Ziefle
    • 1
  • Michael Sedlmair
    • 2
  1. 1.Human-Computer Interaction CenterRWTH Aachen UniversityAachenGermany
  2. 2.Jacobs University BremenBremenGermany

Personalised recommendations