Skip to main content
SpringerLink
Log in

Experiments in Schema-Driven Interpretation of a Natural Scene

  • Conference paper
Digital Image Processing

Part of the book series: NATO Advanced Study Institutes Series ((ASIC,volume 77))

Abstract

The system under development, VISIONS, is an investigation into general issues in the construction of computer vision systems. The goal is to provide an analysis of color images of outdoor scenes, from segmentation (or partitioning) of an image through the final stages of symbolic interpretation of that image. The output of the system is intended to be a symbolic representation of the three-dimensional world depicted in the two-dimensional image, including the naming of objects, their placement in three-dimensional space, and the ability to predict from this representation the rough appearance of the scene from other points of view. Research in segmentation and interpretation has been separated into the development of two major subsystems with quite different methodologies and considerations.

The focus of this paper is upon the interpretation system. The primary emphasis will be on the development of strategies by which several knowledge sources (KSs) can be integrated using expected knowledge stored in structures called 3D and 2D schemas, each of which may be general or specific to the scene under consideration. A series of increasingly more difficult experiments is outlined as an experimental methodology for developing schema-driven (e.g., top-down) control mechanisms; each succeeding experiment will assume a set of weaker constraints, representing image interpretation tasks where a decreasing amount of knowledge of the situation is available. Experimental results show current capabilities of a number of KSs and the effectiveness of a specific 2D schema in the interpretation of a scene.

This research was supported by the Office of Naval Research under Grant N00014-75-C-0459, and the National Science Foundation under Grant MCS79-18209.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. G.J. Agin, “Representation and Description of Curved Objects,” Stanford AI Memo 73, 1972.

    Google Scholar 

  2. G.J. Agin and T.O. Binford, “Computer Description of Curved Objects,” IEEE Transactions on Computers, April 1976, pp. 439–449.

    Google Scholar 

  3. M.A. Arbib, “Parallelism, Slides, Schemas, and Frames,” in Systems: Approaches, Theories, Applications (W.E. Hartnett, Ed.), D. Reidel Publishing Co., 1977, pp. 27–43.

    Chapter  Google Scholar 

  4. R. Bajcsy and M. Tavakoli, “Computer Recognition of Roads from Satellite Pictures,” IEEE Transactions on Systems, Man, and Cybernetics, Vol. SMC-6, September 1976, pp. 623–637.

    Article  Google Scholar 

  5. D.H. Ballard, C.M. Brown, and J.A. Feldman, “An Approach to Knowledge-Directed Image Analysis,” Computer Vision Systems (A. Hanson and E. Riseman, Eds.), Academic Press, pp. 271–281, 1978.

    Google Scholar 

  6. H. Barrow and J.M. Tenenbaum, “MSYS: A System for Reasoning About Scenes,” Technical Note 121, Artificial Intelligence Center, Stanford Research Institute, Menlo Park, CA, April 1976.

    Google Scholar 

  7. H.G. Barrow and J.M. Tenenbaum, “Recovering Intrinsic Scene Characteristics from Images,” Computer Vision Systems (A. Hanson and E. Riseman, Eds.), Academic Press, pp. 3–26, 1978.

    Google Scholar 

  8. B.L. Bullock, “The Necessity for a Theory of Specialized Vision,” Computer Vision Systems (A. Hanson and E. Riseman, Eds.), Academic Press, pp. 27–35, 1978.

    Google Scholar 

  9. L.S. Davis, “Shape Matching Using Relaxation Techniques,” Technical Report 480, Computer Science Center, University of Maryland, College Park, MD, September 1976.

    Google Scholar 

  10. R.O. Duda and P.E. Hart, Pattern Classification and Scene Analysis, John Wiley and Sons, 1973.

    MATH  Google Scholar 

  11. S.A. Dudani and A.L. Luk, “Locating Straight-Line Edge Segments on Outdoor Scenes,” Proc. of Conf. on Pattern Recognition and Image Processing, Troy, NY, June, 1977, pp. 367–377.

    Google Scholar 

  12. C.R. Dyer, A. Rosenfeld, and H. Samet, “Region Representation: Boundary Codes from Quadtrees,” Communications of the ACM, 3, 1980, pp. 171–179.

    Article  Google Scholar 

  13. L.D. Erman and V.R. Lesser, “A Multi-Level Organization for Problem Solving Using Many Diverse Cooperating Sources of Knowledge,” Proc. 4th Inter. Joint Conf. on Artificial Intelligence, Tbilisi, USSR, 1975, pp. 483–490.

    Google Scholar 

  14. J.A. Feldman and Y. Yakimovsky, “Decision Theory and Artificial Intelligence: I. A Semantics-Based Region Analyzer,” Artificial Intelligence, Vol. 5, 1974, pp. 349–371.

    Article  MATH  Google Scholar 

  15. D.P. Friedman, D.C. Dickson, J.J. Fraser, and T.W. Pratt, “GRASPE 1.5 - A Graph Processor and Its Application,” Tech. Report, University of Houston, 1969.

    Google Scholar 

  16. A.R. Hanson and E.M. Riseman, “Preprocessing Cones: A Computational Structure for Scene Analysis,” COINS TR 74C-7, Univ. of Mass., Amherst, September 1974.

    Google Scholar 

  17. A.R. Hanson and E.M. Riseman (Eds.), Computer Vision Systems, Academic Press, 1978.

    Google Scholar 

  18. A.R. Hanson and E.M. Riseman, “Segmentation of Natural Scenes,” in Computer Vision Systems (A.R. Hanson and E.M. Riseman, Eds.), Academic Press, pp. 129–163, 1978.

    Google Scholar 

  19. A.R. Hanson and E.M. Riseman, “VISIONS: A Computer System for Interpreting Scenes,” in Computer Vision Systems (A.R. Hanson and E.M. Riseman, Eds.), Academic Press, pp. 303–333, 1978.

    Google Scholar 

  20. A.R. Hanson, E.M. Riseman and F.C. Glazer, “Edge Relaxation and Boundary Continuity,” in Consistent Labeling Problems in Pattern Recognition (R.M. Haralick, Ed.), Plenum Press, 1980.

    Google Scholar 

  21. A.R. Hanson and E.M. Riseman, “Processing Cones: A Computational Structure for Image Analysis,” in Structured Computer Vision (S. Tanimoto and A. Klinger, Eds.), Academic Press, 1980.

    Google Scholar 

  22. R. Haralick, “Using Perspective Transformations in Scene Analysis,” Technical Report, Electrical Engineering Department, University of Kansas, Lawrence, May 1978.

    Google Scholar 

  23. W.S. Havens, “A Procedural Model of Recognition for Machine Perception,” TR-78-3, Ph.D. Thesis, Department of Computer Science, University of British Columbia, Vancouver, Canada, 1978.

    Google Scholar 

  24. B.K.P. Horn, “Understanding linage Intensities,” Artificial Intelligence, Vol. 8, No. 2, 1977, pp. 201–231.

    Article  MATH  Google Scholar 

  25. R. Kohler, “Reference Manual for the VISIONS Low-Level Image Processing System,” COINS Dept., Univ. of Mass., Amherst, Spring 1979.

    Google Scholar 

  26. K. Konolige, “The ALISP Manual,” Univ. Computing Center, Univ. of Mass., August 1975.

    Google Scholar 

  27. V.R. Lesser and L.D. Erman, “A Retrospective View of the Hearsay-II Architecture,” Proc. Inter. Joint Conf. on Artificial Intelligence, Cambridge, MA, 1977. pp. 790–800.

    Google Scholar 

  28. M.D. Levine, “A Knowledge-Based Computer Vision System,” in Computer Vision Systems (A.R. Hanson and E.M. Riseman, Eds.), Academic Press, 1978, pp. 335–352.

    Google Scholar 

  29. B.T. Lowerre, “The HARPY Speech Recognition Systems,” Ph.D. Thesis, Department of Computer Science, Carnegie-Mellon University, Pittsburgh, PA, 1976.

    Google Scholar 

  30. J.D. Lowrance, “GRASPER 1.0 Reference Manual,” COINS Technical Report 78-20, University of Massachusetts, Amherst, December 1978.

    Google Scholar 

  31. J.D. Lowrance, “Dependency-Graph Models of Evidential Support,” Ph.D. Dissertation, COINS Dept., Univ. of Mass., Amherst, expected June 1980.

    Google Scholar 

  32. A.K. Mackworth, “Vision Research Strategy: Black Magic, Metaphors, Mechanisms, Miniworlds, and Maps,” in Computer Vision Systems (A.R. Hanson and E.M. Riseman, Eds.), Academic Press, 1978, pp. 53–60.

    Google Scholar 

  33. D. Marr, “Early Processing of Visual Information,” Phil. Trans. Roy. Soc. B275, 1976, pp. 483–524.

    Google Scholar 

  34. D. Marr and H. K. Nishihara, “Representation and Recognition of the Spatial Organization of Three-Dimensional Shapes,” Proc. Roy. Soc. B.200, 1977, pp. 269–294.

    Google Scholar 

  35. D. Marr, “Representing Visual Information,” in Computer Vision Systems (A.R. Hanson and E.M. Riseman, Eds.), Academic Press, 1978, pp. 61–80.

    Google Scholar 

  36. M. Minsky, “A Framework for Representing Knowledge,” in The Psychology of Computer Vision (P. Winston, Ed.), McGraw-Hill, 1975, pp. 211–277.

    Google Scholar 

  37. P. Nagin, “Studies in Image Segmentation Algorithms Based on Histogram Clustering and Relaxation,” COINS Technical Report 79-15 and Ph.D. Dissertation, Univ. of Mass., Amherst, September 1979.

    Google Scholar 

  38. R. Nevatia, Computer Analysis of Scenes of 3-Dimensional Curved Objects, Birkhauser-Verlag, Basel, Switzerland, 1976.

    Google Scholar 

  39. R. Nevatia and T.O. Binford, “Description and Recognition of Curved Objects,” Artificial Intelligence, Vol. 8, 1977, pp. 77–98.

    Article  MATH  Google Scholar 

  40. R. Nevatia, “Characterization and Requirements of Computer Vision Systems,” in Computer Vision Systems (A.R. Hanson and E.M. Riseman, Eds.), Academic Press, 1978, pp. 81–87.

    Google Scholar 

  41. J.R. Newman, The Universal Encyclopedia of Mathematics, The New American Library, July 1965.

    Google Scholar 

  42. K.J. Overton and T.E. Weymouth, “A Noise Reducing Preprocessing Algorithm,” Proc. of Pattern Recognition and Image Processing Conference, Chicago, Illinois, August 1979, pp. 498–507.

    Google Scholar 

  43. C.C. Parma, A.R. Hanson and E.M. Riseman, “Experiments in Schema-Driven Interpretation of a Natural Scene,” COINS TR 80-10, Univ. of Mass., Amherst, April 1980.

    Google Scholar 

  44. T. Pratt and D. Friedman, “A Language Extension for Graph Processing and Its Formal Semantics,” Communications of the ACM, 4, 1971.

    Google Scholar 

  45. J. Prager, “Analysis of Static and Dynamic Scenes” Ph.D. Dissertation, COINS Dept., Univ. of Mass., Amherst, March 1979.

    Google Scholar 

  46. J. Prager, “Extracting and Labeling Boundary Segments in Natural Scenes,” IEEE Trans. Pattern Analysis and Machine Intelligence, Vol. PAMI-2, January 1980, pp. 16–27.

    Article  Google Scholar 

  47. E.M. Riseman and A.R. Hanson, “Design of a Semantically Directed Vision Processor,” COINS TR 74C-1, Univ. of Mass., Amherst, January 1974.

    Google Scholar 

  48. E.M. Riseman and M.A. Arbib, “Computational Techniques in the Visual Segmentation of Static Scenes,” Computer Graphics and Image Processing, 6, 1977, pp. 221–276.

    Article  Google Scholar 

  49. L.G. Roberts, “Machine Perception of Three-Dimensional Solids,” Optical and Electro-Optical Information Processing (J.T. Tippet et al., Eds.), MIT Press, 1965.

    Google Scholar 

  50. A. Rosenfeld, R.A. Hummel and S.W. Zucker, “Scene Labelling by Relaxation Operations,” IEEE Trans. Systems, Man, and Cybernetics, 6, 1976, pp. 420–433.

    Article  MathSciNet  MATH  Google Scholar 

  51. S.M. Rubin and R. Reddy, “The Locus Model of Search and Its Use in Image Interpretation,” Proc. of Fifth IJCAI, Cambridge, MA, August 1977.

    Google Scholar 

  52. T. Sakai, T. Kanade, and Y. Ohta, “Model-Based Interpretation of Outdoor Scenes,” Third Int. Joint Conf. on Pattern Recognition, Coronado, CA, November 1976, pp. 581–585.

    Google Scholar 

  53. H. Samet, “Region Representation: Quadtrees from Boundary Codes,” Communications of the ACM, 3. 1980. pp. 163–170.

    Article  Google Scholar 

  54. R.C. Schank and R. Abelson, “Scripts, Plans, and Knowledge,” Proc. of Fourth IJCAI, Tbilisi, 1975, pp. 151–158.

    Google Scholar 

  55. R.C. Schank and R.P. Abelson, Goals, Plans, Scripts and Understanding: An Enquiry into Human Knowledge Structures, Erlbaum Press, NJ, 1977.

    Google Scholar 

  56. R.C. Schank, Interdisciplinary Conference, Jackson, Wyoming, January 1979.

    Google Scholar 

  57. Y. Shirai, “Recognition of Real-World Objects Using Edge Cues,” in Computer Vision Systems (A.R. Hanson and E.M. Riseman, Eds.), Academic Press, 1978, pp. 353–362.

    Google Scholar 

  58. S.L. Tanimoto, “Regular Hierarchical Image and Processing Structures in Machine Vision,” in Computer Vision Systems (A.R. Hanson and E.M. Riseman, Eds.), Academic Press, 1978, pp. 165–174.

    Google Scholar 

  59. S. Tanimoto and A. Klinger (Eds.), Structured Computer Vision, Academic Press, 1980.

    Google Scholar 

  60. J.M. Tenenbaum and H.G. Barrow, “Experiments in Interpretation-Guided Segmentation,” Artificial Intelligence, 8, No. 3, 1977, pp. 241–274.

    Article  Google Scholar 

  61. L. Uhr, “Layered ‘Recognition Cone’ Networks That Preprocess, Classify, and Describe,” IEEE Trans. Computers, 1972, pp. 758–768.

    Google Scholar 

  62. L. Uhr, “‘Recognition Cones,’ and Some Test Results; The Imminent Arrival of Well-Structured Parallel-Serial Computers; Positions, and Positons on Positions,” in Computer Vision Systems (A.R. Hanson and E.M. Riseman, Eds.), Academic Press, 1978, pp. 363–377.

    Google Scholar 

  63. T. Williams and J. Lowrance, “Model-Building in the VISIONS High Level System,” COINS Technical Report 77-1, Univ. of Mass., Amherst, January 1977.

    Google Scholar 

  64. T. Williams, Ph.D. Dissertation (in preparation). COINS Dept., Univ. of Mass., expected June 1980.

    Google Scholar 

  65. D. Waltz, “Understanding Line Drawings of Scenes with Shadows,” in The Psychology of Computer Vision (P. Winston, Ed.), McGraw-Hill, 1975, pp. 19–91.

    Google Scholar 

  66. Y. Yakimovsky and J.A. Feldman, “A Semantics-Based Decision Theory Region Analyzer,” Proc. IJCAI-3, August 1973, pp. 580–588.

    Google Scholar 

  67. B. York, “A Primer on Splines,” COINS TR 79-5, Univ. of Mass., Amherst, Mass., March 1979.

    Google Scholar 

  68. B. York, Ph.D. Dissertation (in preparation), COINS Dept., Univ. of Mass., expected June 1980.

    Google Scholar 

  69. S.W. Zucker, R.A. Hummel, and A. Rosenfeld, “An Application of Relaxation Labelling to Line and Curve Enhancement,” IEEE Transactions on Computers, Vol. C-26, April 1977, pp. 394–403.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer and Information Science Department, University of Massachusetts, Amherst, Massachusetts, 01003, USA

    Cesare C. Parma, Allen R. Hanson & Edward M. Riseman

Authors
  1. Cesare C. Parma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Allen R. Hanson
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Edward M. Riseman
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Institut de Programmation, Université Pierre et Marie Curie, Paris VI, France

    J. C. Simon

  2. Virginia Polytechnic Institute and State University, Blacksburg, USA

    R. M. Haralick

Rights and permissions

Reprints and permissions

Copyright information

© 1981 D. Reidel Publishing Company

About this paper

Cite this paper

Parma, C.C., Hanson, A.R., Riseman, E.M. (1981). Experiments in Schema-Driven Interpretation of a Natural Scene. In: Simon, J.C., Haralick, R.M. (eds) Digital Image Processing. NATO Advanced Study Institutes Series, vol 77. Springer, Dordrecht. https://doi.org/10.1007/978-94-009-8543-8_25

Download citation

  • DOI: https://doi.org/10.1007/978-94-009-8543-8_25

  • Publisher Name: Springer, Dordrecht

  • Print ISBN: 978-94-009-8545-2

  • Online ISBN: 978-94-009-8543-8

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation