Combining multiple views and temporal associations for 3-D object recognition

  • Amin Massad
  • Bärbel Mertsching
  • Steffen Schmalz
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 1407)


This article describes an architecture for the recognition of three-dimensional objects on the basis of viewer centred representations and temporal associations. Considering evidence from psychophysics, neurophysiology, as well as computer science we have decided to use a viewer centred approach for the representation of three-dimensional objects. Even though this concept quite naturally suggests utilizing the temporal order of the views for learning and recognition, this aspect is often neglected. Therefore we will pay special attention to the evaluation of the temporal information and embed it into the conceptual framework of biological findings and computational advantages. The proposed recognition system consists of four stages and includes different kinds of artificial neural networks: Preprocessing is done by a Gabor-based wavelet transform. A Dynamic Link Matching algorithm, extended by several modifications, forms the second stage. It implements recognition and learning of the view classes. The temporal order of the views is recorded by a STORE network which transforms the output for a presented sequence of views into an item- and-order coding. A subsequent Gaussian-ARTMAP architecture is used for the classification of the sequences and for their mapping onto object classes by means of supervised learning. The results achieved with this system show its capability to autonomously learn and to recognize considerably similar objects. Furthermore the given examples illustrate the benefits for object recognition stemming from the utilization of the temporal context. Ambiguous views become manageable and a higher degree of robustness against misclassifications can be accomplished.


Object Recognition Temporal Order Object Class Temporal Association View Class 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Biederman, I. (1985): Human image understanding: Recent research and a theory. Comput. Vision Graphics Image Processing 32: 29–73.CrossRefGoogle Scholar
  2. Bradski, G.; Carpenter, G. and Grossberg, S. (1992): Working memory networks for learning temporal order with application to three-dimensional visual object recognition. Neural Comput. 4: 270–286.Google Scholar
  3. Bradski, G. and Grossberg, S. (1993): Fast learning VIEWNET architectures for recognizing 3-D objects from multiple 2-D views. Technical Report CAS/CNS-TR-93-053, Boston Univ., Boston, MA.Google Scholar
  4. Carpenter, G.; Grossberg, S.; Markuzon, N.; Reynolds, J. and Rosen, D. (1992): Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps. IEEE Trans. Neural Networks 3(5): 698–713.CrossRefGoogle Scholar
  5. Darrell, T. and Pentland, A. (1993): Recognition of space-time gestures using a distributed representation. In R. Mammone (ed.), Artificial neural networks for speech and vision. London: Chapman & Hall, pp. 502–519.Google Scholar
  6. Edelman, S. and Weinshall, D. (1991): A self-organizing multiple-view representation of 3D objects. Biol. Cybern. 64: 209–219.CrossRefGoogle Scholar
  7. Grossberg, S. and Bradski, G. (1995): VIEWNET architectures for invariant 3-D object recognition from multiple views. In B. Bouchon-meunier; R. Yager and L. Zadeh (eds.), Fuzzy logic and soft computing. Singapore: World Scientific Publishing.Google Scholar
  8. Koenderink, J. and van Doorn, A. (1979): The internal representation of solid shape with respect to vision. Biol. Cybern. 32: 211–216.zbMATHCrossRefGoogle Scholar
  9. Konen, W.; Maurer, T. and von der Malsburg, C. (1994): A fast link matching algorithm for invariant pattern recognition. Neural Networks 7: 1019–1030.zbMATHCrossRefGoogle Scholar
  10. Lowe, D. G. (1986): Perceptual organization and visual recognition. Boston: Kluwer.Google Scholar
  11. Marr, D. and Nishihara, H. K. (1978): Representation and recognition of the spatial organization of three-dimensional shapes. Proc. R. Soc. Lond. 200: 269–294.CrossRefGoogle Scholar
  12. Matsakis, Y.; Berthoz, A.; Lipschits, M. and Gurfinkel, V. (1990): Mental rotation of three-dimensional shapes in microgravity. In Proc. Fourth European Symposium on Life Sciences in Space. ESA, pp. 625–629.Google Scholar
  13. Metzler, J. and Shepard, R. (1974): Transformational studies of the internal representaion of three-dimensional objects. In R. Solso (ed.), Theories in cognitive psychology: The Loyola Symposium. Erlbaum.Google Scholar
  14. Miyashita, Y. (1988): Neuronal correlate of visual associative long-term memory in the primate temporal cortex. Nature 335: 817–820.CrossRefGoogle Scholar
  15. Perrett, D.; Harries, M.; Benson, P.; Chitty, A. and Mistlin, A. (1990): Retrieval of structure from rigid and biological motion: An analysis of the visual responses of neurones in the Macaque temporal cortex. In A. Blake and T. Troscianko (eds.), AI and the Eye, chap. 8. Wiley & Sons, pp. 181–200.Google Scholar
  16. Perrett, D.; Oram, M.; Harries, M.; Bevan, R.; Hietanen, J.; Benson, P. and Thomas, S. (1991): Viewer-centred and object-centred coding of heads in the macaque tempral cortex. Experimental Brain Research 86: 159–173.CrossRefGoogle Scholar
  17. Poggio, T. and Edelman, S. (1990): A network that learns to recognize three-dimensional objects. Nature 343: 263–266.CrossRefGoogle Scholar
  18. Sakai, K. and Miyashita, Y. (1991): Neural organization for the long-term memory of paired associates. Nature 354: 152–155.CrossRefGoogle Scholar
  19. Sarle, W. (1995): Why statisticians should not FART. neural/fart.doc.Google Scholar
  20. Seibert, M. and Waxman, A. (1992): Adaptive 3-D object recognition from multiple views. IEEE Trans. Pattern Anal. and Machine Intel. 14(2): 107–124.CrossRefGoogle Scholar
  21. Stryker, M. P. (1991): Temporal associations. Nature 354: 108–109.CrossRefGoogle Scholar
  22. Sumi, S. (1984): Upside-down presentation of the Johansson moving light-spot pattern. Perception 13: 283–286.Google Scholar
  23. Tanaka, K. (1996): Inferotemporal cortex and object vision. Annu. Rev. Neurosci. 19: 109–139.CrossRefGoogle Scholar
  24. Thompson, D. W. and Mundy, J. L. (1987): Three dimensional model matching from an unconstrained viewpoint. In Proc. IEEE Int. Conf. Robotics and Automation. Raleigh, NC: IEEE, pp. 208–220.Google Scholar
  25. Ullman, S. and Basri, R. (1991): Recognition by linear combinations of models. IEEE Trans. Pattern Anal. and Machine Intel. 13(10): 992–1006.CrossRefGoogle Scholar
  26. von der Malsburg, C. (1981): The correlation theory of brain function. Internal report, Max-Planck-Institut für Biophysikalische Chemie, Göttingen.Google Scholar
  27. von der Malsburg, C. and Reiser, K. (1995): Pose invariant object recognition in a neural system. In Proc. Int. Conf. on Artificial Neural Networks ICANN. pp. 127–132.Google Scholar
  28. Wallis, G. (in press): Temporal order in human object recognition learning. To appear in Journal of Biological Systems.Google Scholar
  29. Williamson, J. (1996): Gaussian ARTMAP: A neural network for fast incremental learning of noisy multidimensional maps. Neural Networks 9(5): 881–897.CrossRefGoogle Scholar
  30. Wiskott, L. and von der Malsburg, C. (1996): Face recognition by Dynamic Link Matching. Internal Report IR-INI 96-05, Inst. f. Neuroinformatik, Ruhr-Univ. Bochum.Google Scholar
  31. Würtz, R. (1994): Multilayer dynamic link networks for establishing image point correspondences and visual object recognition. Ph.D. thesis, Ruhr-Univ. Bochum.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1998

Authors and Affiliations

  • Amin Massad
    • 1
  • Bärbel Mertsching
    • 1
  • Steffen Schmalz
    • 1
  1. 1.Dep. of Computer Science, AG IMAUniversity of HamburgHamburgGermany

Personalised recommendations