Schema-Driven Influences in Recovering 3-D Shape from Motion in Human and Computer Vision

Part of the Advances in Computer Vision and Pattern Recognition book series (ACVPR)


To investigate the role of prior knowledge in perceiving the hollow-face illusion, we set out to correlate human performance with that of a computational model that tracks facial features. Toward this goal, we prepared a video of a thin human mask, realistically painted on both the convex and concave sides, that rotated by 360 degrees. It is well known that when humans view the rotating hollow part of the mask, they perceive it as a convex face that rotates in the opposite direction. We submitted this video as input to DeCarlo & Metaxas’s (Int. J. Comput. Vis. 38(2):99–127, 2000) face tracking algorithm that uses a combination of optical flow and feature alignment to track moving faces; importantly, the algorithm uses a schema of a 3-D convex face. The algorithm was fooled, as humans are, into misperceiving a concave face as a convex face rotating in the opposite direction. Notably, this is a significant case where a computer vision algorithm “experiences” an illusion. However, when the convex-face schema was replaced with a schema that allowed both convex and concave faces, the algorithm tracked the mask accurately. The similarity in the responses of the computer vision algorithm and human observers reinforces the hypothesis that a built-in convex 3D face schema imposes a depth inversion for concave faces that in turn forces the false interpretation of motion signals.


Motion Parallax Computer Vision Algorithm Animation Sequence Kinetic Depth Effect Convex Face 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



We would like to thank Manpreet Kaur for conducting experiments on the hollow-mask illusion with human observers. We thank Christopher Tyler who reviewed the manuscript and offered valuable suggestions.


  1. 1.
    Adams WJ, Graf EW, Ernst MO (2004) Experience can change the ‘light-from-above’ prior. Nat Neurosci 7:1057–1058 CrossRefGoogle Scholar
  2. 2.
    Aglioti S, DeSouza JFX, Goodale MA (1995) Size contrast illusions deceive the eye but not the hand. Curr Biol 5:679–685 CrossRefGoogle Scholar
  3. 3.
    Ahissar M, Hochstein S (2004) The reverse hierarchy theory of perceptual learning. Trends Cogn Sci 8(10):457–464 CrossRefGoogle Scholar
  4. 4.
    Banks MS, Backus BT (1998) Extra-retinal and perspective cues cause the small range of the induced effect. Vis Res 38:187–194 CrossRefGoogle Scholar
  5. 5.
    Barlow HB (1997) The knowledge used in vision and where it comes from. Philos Trans R Soc Lond B, Biol Sci 352(1358):1141–1147 CrossRefGoogle Scholar
  6. 6.
    Berbaum K, Bever T, Chung CS (1983) Light source position in the perception of object shape. Perception 12:411–416 CrossRefGoogle Scholar
  7. 7.
    Berbaum K, Bever T, Chung CS (1984) Extending the perception of shape from known to unknown shading. Perception 13:479–488 CrossRefGoogle Scholar
  8. 8.
    Caudek C, Fantoni C, Domini F (2011) Bayesian modeling of perceived surface slant from actively-generated and passively-observed optic flow. PLoS ONE 6(4):e18731 CrossRefGoogle Scholar
  9. 9.
    Cook ND, Hayashi T, Amemiya T, Suzuki K, Leumann L (2002) Effects of visual field inversions on the reverse-perspective illusion. Perception 31:1147–1151 CrossRefGoogle Scholar
  10. 10.
    DeCarlo D, Metaxas D (2000) Optical flow constraints on deformable models with applications to face tracking. Int J Comput Vis 38(2):99–127 MATHCrossRefGoogle Scholar
  11. 11.
    Dima D, Roiser JP, Dietrich DE, Bonnemann C, Lanfermann H, Emrich HM, Dillo W (2009) Understanding why patients with schizophrenia do not perceive the hollow-mask illusion using dynamic causal modeling. NeuroImage 46:1180–1186 CrossRefGoogle Scholar
  12. 12.
    Dima D, Dillo W, Bonnemann C, Emrich HM, Dietrich DE (2011) Reduced P300 and P600 amplitude in the hollow-mask illusion in patients with schizophrenia. Psychiatry Res Neuroimaging 191:145–151 CrossRefGoogle Scholar
  13. 13.
    Emrich HM, Leweke FM, Schneider U (1997) Towards a cannabinoid hypothesis of schizophrenia: cognitive impairments due to a dysregulation of the endogenous cannabinoid system. Pharmacol Biochem Behav 56:803–807 CrossRefGoogle Scholar
  14. 14.
    Emrich HM, Weber MM, Wendl A, Zihl J, Von Meyer L, Hanisch W (1991) Reduced binocular depth inversion as an indicator of cannabis-induced censorship impairment. Pharmacol Biochem Behav 40:689–690 CrossRefGoogle Scholar
  15. 15.
    Feldman J, Singh M (2006) Bayesian estimation of the shape skeleton. Proc Natl Acad Sci USA 103(47):18014–18019 MathSciNetMATHCrossRefGoogle Scholar
  16. 16.
    Franz VH, Gegenfurtner KR, Bulthoff HH, Fahle M (2000) Grasping visual illusions: no evidence for a dissociation between perception and action. Psychol Sci 11:20–25 CrossRefGoogle Scholar
  17. 17.
    Gilbert CD, Sigman M (2007) Brain states: top-down influences in sensory processing. Neuron 54:677–696 CrossRefGoogle Scholar
  18. 18.
    Goldberg I, Harel M, Malach R (2006) When the brain loses its self: prefrontal inactivation during sensorimotor processing. Neuron 50:329–339 CrossRefGoogle Scholar
  19. 19.
    Goldreich D, Peterson MA (2012) A Bayesian observer replicates convexity context effects in figure-ground perception. Seeing Perceiving 25(3–4):365–395. doi: 10.1163/187847612X634445 CrossRefGoogle Scholar
  20. 20.
    Gregory RL (1968) Perceptual illusions and brain models. Proc R Soc B 171:279–296 CrossRefGoogle Scholar
  21. 21.
    Gregory RL (1970) The intelligent eye. McGraw-Hill, New York, pp 126–131 Google Scholar
  22. 22.
    Gregory RL (1980) Perceptions as hypotheses. Philos Trans R Soc Lond B, Biol Sci 290:181–197 CrossRefGoogle Scholar
  23. 23.
    Gregory RL (1997) Knowledge in perception and illusion. Philos Trans R Soc Lond B, Biol Sci 352:1121–1128 CrossRefGoogle Scholar
  24. 24.
    Gregory RL (2005) The Medawar lecture 2001 knowledge for vision: vision for knowledge. Philos Trans R Soc Lond B, Biol Sci 360(1458):1231–1251 CrossRefGoogle Scholar
  25. 25.
    Held RT, Cooper EA, Banks MS (2012) Blur and disparity are complementary cues to depth. Curr Biol 22:426–431 CrossRefGoogle Scholar
  26. 26.
    Helmholtz H (1910/1867). Handbuch der Physiologischen Optik, vol 3. Voss Google Scholar
  27. 27.
    Hill H, Bruce V (1993) Independent effects of lighting, orientation, and stereopsis on the hollow-face illusion. Perception 22(8):887–897 CrossRefGoogle Scholar
  28. 28.
    Hill H, Bruce V (1994) A comparison between the hollow-face and ‘hollow-potato’ illusions. Perception 23:1335–1337 CrossRefGoogle Scholar
  29. 29.
    Hill H, Johnston A (2007) The hollow-face illusion: object-specific knowledge, general assumptions or properties of the stimulus? Perception 36:199–223 CrossRefGoogle Scholar
  30. 30.
    Hochstein S, Ahissar M (2002) View from the top: hierarchies and reverse hierarchies in the visual system. Neuron 36:791–804 CrossRefGoogle Scholar
  31. 31.
    Jain A, Zaidi Q (2011) Discerning nonrigid 3D shapes from motion cues. Proc Natl Acad Sci USA 108(4):1663–1668 CrossRefGoogle Scholar
  32. 32.
    Jones MJ, Sinha P, Vetter T, Poggio T (1997) Top-down learning of low-level vision tasks. Curr Biol 7:991–994 CrossRefGoogle Scholar
  33. 33.
    Keane BP, Silverstein SM, Wang Y, Papathomas TV (2013) Reduced depth illusions in schizophrenia: Evidence for a weakened, state-dependent convexity prior. J Abnormal Psych 122(2):506–512 CrossRefGoogle Scholar
  34. 34.
    Kersten D, Yuille A (2003) Bayesian models of object perception. Curr Opin Neurobiol 13:1–9 CrossRefGoogle Scholar
  35. 35.
    Kersten D, Mamassian P, Yuille A (2004) Object perception as Bayesian inference. Annu Rev Psychol 55:271–304 CrossRefGoogle Scholar
  36. 36.
    Kleffner DA, Ramachandran VS (1992) On the perception of shape from shading. Percept Psychophys 52:18–36 CrossRefGoogle Scholar
  37. 37.
    Koethe D, Gerth CW, Neatby MA, Haensel A, Thies M, Schneider U, Emrich HM, Klosterkotter J, Schultze-Lutter F, Leweke FM (2006) Disturbances of visual information processing in early states of psychosis and experimental delta-9-tetrahydrocannabinol altered states of consciousness. Schizophr Res 88:142–150 CrossRefGoogle Scholar
  38. 38.
    Krekelberg B, Lappe M (2001) Neuronal latencies and the position of moving objects. Trends Neurosci 24:335–339 CrossRefGoogle Scholar
  39. 39.
    Langer MS, Bülthoff HH (2001) A prior for global convexity in local shape-from-shading. Perception 30:403–410 CrossRefGoogle Scholar
  40. 40.
    Palmer S (1999) Vision science: from photons to phenomenology. MIT Press, Cambridge Google Scholar
  41. 41.
    Papathomas TV (1999) The brain as a hypothesis-constructing-and-testing agent. In: LePore E, Pylyshyn Z (eds) What is cognitive science? Blackwell, Oxford, pp 230–247 Google Scholar
  42. 42.
    Papathomas TV (2007) Art pieces that ‘move’ in our minds—an explanation of illusory motion based on depth reversal. Spat Vis 21:79–95 CrossRefGoogle Scholar
  43. 43.
    Papathomas TV, Bono L (2004) Experiments with a hollow mask and a reverspective: top-down influences in the inversion effect for 3-d stimuli. Perception 33:1129–1138 CrossRefGoogle Scholar
  44. 44.
    Pavlidis T (2012) Personal communication. December 15, 2012 Google Scholar
  45. 45.
    Pizlo Z (2001) Perception viewed as an inverse problem. Vis Res 41(24):3145–3161 CrossRefGoogle Scholar
  46. 46.
    Poggio T, Torre V, Koch C (1985) Computational vision and regularization theory. Nature 317:314–319 CrossRefGoogle Scholar
  47. 47.
    Purves D, Lotto RB (2003) Why we see what we do. Sinauer, Sunderland Google Scholar
  48. 48.
    Ramachandran VS (1988) Perception of shape from shading. Nature 331:163–166 CrossRefGoogle Scholar
  49. 49.
    Ramachandran VS (1995) Anosognosia in parietal lobe syndrome. Conscious Cogn 4(1):22–51 CrossRefGoogle Scholar
  50. 50.
    Regan D, Erkelens CJ, Collewijn H (1986) Necessary conditions for the perception of motion in depth. Investig Ophthalmol Vis Sci 27:584–597 Google Scholar
  51. 51.
    Richards W (2009) Configuration stereopsis: a new look at the depth-disparity relation. Spat Vis 22(1):91–103 CrossRefGoogle Scholar
  52. 52.
    Rock I (1983) The logic of perception. MIT Press, Cambridge Google Scholar
  53. 53.
    Rogers B (2009) Motion parallax as an independent cue for depth perception: a retrospective. Perception 38(6):907–911 CrossRefGoogle Scholar
  54. 54.
    Rogers B, Graham M (1979) Motion parallax as an independent cue for depth perception. Perception 8(2):125–134 CrossRefGoogle Scholar
  55. 55.
    Rogers BJ, Gyani A (2010) Binocular disparities, motion parallax, and geometric perspective in Patrick Hughes’s ‘reverspectives’: theoretical analysis and empirical findings. Perception 39:330–348 CrossRefGoogle Scholar
  56. 56.
    Semple DM, Ramsden F, McIntosh AM (2003) Reduced binocular depth inversion in regular cannabis users. Pharmacol Biochem Behav 75(4):789–793 CrossRefGoogle Scholar
  57. 57.
    Schneider U, Borsutzky M, Seifert J, Leweke FM, Huber TJ, Rollnik JD, Emrich HM (2002) Reduced binocular depth inversion in schizophrenic patients. Schizophr Res 53:101–108 CrossRefGoogle Scholar
  58. 58.
    Schneider U, Dietrich DE, Sternemann U, Seeland I, Gielsdorf D, Huber TJ, Becker H, Emrich HM (1998) Reduced binocular depth inversion in patients with alcoholism. Alcohol Alcoholism 33:168–172 CrossRefGoogle Scholar
  59. 59.
    Schneider U, Leweke FM, Sternemann U, Weber MM, Emrich HM (1996) Visual 3D illusion: a systems-theoretical approach to psychosis. Eur Arch Psychiatry Clin Neurosci 246:256–260 CrossRefGoogle Scholar
  60. 60.
    Sherman A, Papathomas TV, Jain A, Keane BP (2011) The roles of perspective, angle polarity, stereo and motion parallax in perceiving 3D objects. Seeing Perceiving 25:263–285 CrossRefGoogle Scholar
  61. 61.
    Slyce J (2011) Patrick Hughes: perverspective, 3rd edn. Momentum, London Google Scholar
  62. 62.
    Sperling G, Landy MS (1989) Kinetic depth effect and identification of shape. J Exp Psychol Hum Percept Perform 15(4):826–840 CrossRefGoogle Scholar
  63. 63.
    Sternemann U, Schneider U, Leweke FM, Bevilacqua CM, Dietrich DE, Emrich HM (1997) Pro-psychotic change of binocular depth inversion by sleep deprivation. Nervenarzt 68(7):593–596. [Article in German] CrossRefGoogle Scholar
  64. 64.
    Tyler CW, Kontsevich LL (1995) Mechanisms of stereoscopic processing: stereoattention and surface perception in depth reconstruction. Perception 24(2):127–153 CrossRefGoogle Scholar
  65. 65.
    Uttal WR, Liu N, Kalki J (1996) An integrated computational model of three-dimensional vision. Spat Vis 9(4):393–422 CrossRefGoogle Scholar
  66. 66.
    Vishwanath D, Blaser E (2010) Retinal blur and the perception of egocentric distance. J Vis 10(10):26 CrossRefGoogle Scholar
  67. 67.
    Wade NJ, Hughes P (1999) Fooling the eyes: trompe l’oeil and reverse perspective. Perception 28:1115–1119 CrossRefGoogle Scholar
  68. 68.
    Welchman AE, Harris JM, Brenner E (2009) Extra-retinal signals support the estimation of 3D motion. Vis Res 49:782–789 CrossRefGoogle Scholar
  69. 69.
    Westheimer G (2008) Illusions in the spatial sense of the eye: geometrical-optical illusions and the neural representation of space. Vis Res 48(20):2128–2142 CrossRefGoogle Scholar
  70. 70.
    Yellott JI Jr (1981) Binocular depth inversion. Sci Am 245:118–125 CrossRefGoogle Scholar
  71. 71.
    Yuille A, Kersten D (2006) Vision as Bayesian inference: analysis by synthesis? Trends Cogn Sci 10:301–308 CrossRefGoogle Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  1. 1.Department of Biomedical Engineering and Center for Cognitive ScienceRutgers UniversityPiscatawayUSA
  2. 2.Department of Computer Science and Center for Cognitive ScienceRutgers UniversityPiscatawayUSA

Personalised recommendations