Skip to main content
SpringerLink
Log in

The Issues, Analysis, and Interpretation of Multi-Sensor Images

  • Conference paper
Book cover Multisensor Fusion for Computer Vision

Part of the book series: NATO ASI Series ((NATO ASI F,volume 99))

Abstract

Past research in computer vision has shown that image interpretation is a highly underconstrained task. Information fusion from multiple cues from the same image and from multiple views using the same modality have been marginally successful. Recently the fusion of information from different modalities of sensing has been studied to further constrain the interpretation. This paper presents an overview of approaches developed for image segmentation and analysis using multi-sensor fusion. We present examples of three systems using different modalities. These examples include a system for image segmentation and interpretation using ladar (laser radar) and thermal images, a system using registered thermal and visual images for surface heat flux analysis, and an image synthesis system that generates visual and thermal images based on the internal heat flow in objects.

This research was supported in part by the DoD Joint Service Electronics Program through the Air Force Office of Scientific Research (AFSC) Contract F49620-86-C-0045 and by the Army Research Office under contract DAAL03-87-K-0089.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. N. Nandhakumar, “Application of artificial Intelligence to multisensory vision,” invited talk at IEEE Computer Society Workshop on Artificial Intelligence for Computer Vision, San Diego, June 5, 1989.

    Google Scholar 

  2. B. K. P. Horn, “Robot Vision,” MIT Press, Cambridge, Massachusetts, 1986.

    Google Scholar 

  3. M. L. Moerdler and J. R. Kender, “An approach to the fusion of multiple shape from texture algorithms,” The Proceedings of IEEE Workshop on Spatial Reasoning and Multi-Sensor Fusion, 1987, pp. 272–281.

    Google Scholar 

  4. W. E. L. Crimson, From Images to Surfaces: A Computational Study of the Human Early Vision System, MIT Press, Cambridge, Massachusetts, 1981.

    Google Scholar 

  5. U. Dhond and J. K. Aggarwal, “Structure from stereo: a review,” IEEE Transactions on System, Man, and Cybernetics, SMC-19, no. 6, 1989, pp. 1489–1510.

    Google Scholar 

  6. J. K. Aggarwal and N. Nandhakumar, “On the computation of motion from sequences of images — a review,” Proceedings of the IEEE, vol. 76, no. 8, August 1988, pp. 917–935.

    Article  Google Scholar 

  7. J. K. Aggarwal and Y. F. Wang, “Analysis of a sequence of images using point and line correspondences,” The Proceedings of IEEE Robotics and Automation Conference, March 31-April 3, 1987, pp. 1275–1280.

    Google Scholar 

  8. C. W. Tong, S. K. Rogers, J. P. Mills, and M. K. Kabrisky, “Multisensor data fusion of laser radar and forward looking infrared (FLIR) for target segmentation and enhancement,” Proceedings of SPIE, vol. 782, pp. 10–19.

    Google Scholar 

  9. P. Allen, “A framework for implementing multi-sensor robotic tasks,” The Proceedings of DARPA Image Understanding Workshop, 1987, pp. 392–398.

    Google Scholar 

  10. A. K. Jain and R. Hoffman, “Evidence-based recognition of 3-D objects,” IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-10, no. 6, November 1988, pp. 783–802.

    Google Scholar 

  11. Y. F. Wang and J. K. Aggarwal, “On modeling 3D objects using multiple sensory data,” The Proceedings of IEEE Robotics and Automation Conference, March 31-April 3, 1987, pp. 1098–1103.

    Google Scholar 

  12. Y. F. Wang, A. Mitiche, and J. K. Aggarwal, “Computation of surface orientation and structure of objects using grid coding,” IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-9, no. 1, January 1987, pp. 129–137.

    Google Scholar 

  13. C. H. Chien and J. K. Aggarwal, “Volume surface Octree for the Representation of 3-D Objects,” Computer Vision, Graphics, and Image Processing, vol. 36, 1986, pp. 100–113.

    Google Scholar 

  14. C. H. Chien and J. K. Aggarwal, “Identification of 3D objects from multiple silhouettes using quadtree octree,” Computer Vision, Graphics, and Image Processing, vol. 36, 1986, pp. 256–273.

    Google Scholar 

  15. C. H. Chien and J. K. Aggarwal, “Reconstruction and recognition of 3D objects from occluding contours and silhouettes,” Ph.D. Dissertation, University of Texas at Austin, May 1987.

    Google Scholar 

  16. C. H. Chien and J. K. Aggarwal, “Computation of Volume surface Octree from Contours and Silhouettes of Multiple Views,” Proc. Comp. Vision and Pattern Recog. Conf., Miami Beach, FL, 1986, pp. 220–225.

    Google Scholar 

  17. C. H. Chien, Y. B. Sim and J. K. Aggarwal, “Generation of Volume Surface Octree from Range Data,” Proc. Comp. Vision and Pattern Recog. Conf., Ann Arbor, MI, 1988, pp. 254–260.

    Google Scholar 

  18. B. Gil, A. Mitiche, and J. K. Aggarwal, “Experiments in combining intensity and range edge maps,” Computer Vision, Graphics, and Image Processingvol. 21, 1983, pp. 395–411.

    Google Scholar 

  19. M. J. Magee, B. A. Boyter, C. Chien, and J. K. Aggarwal, “Experiments in intensity guided range sensing recognition of three-dimensional objects,” IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-7, no. 6, November 1985, pp. 629–637.

    Google Scholar 

  20. J. K. Aggarwal and N. Nandhakumar, “Multisensor fusion for automatic scene interpretation - research issues and directions,” Proceedings NSF Workshop on Range Image Processing, edit. R. Jain, March 1988.

    Google Scholar 

  21. P. J. Besl and R. C. Jain, “Segmentation Through Variable-Order Surface Fitting,” IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-10, no. 2, March 1988, pp. 167–192.

    Google Scholar 

  22. C. G. Bachman, Laser Radar Systems and Techniques, Artech House, Dedham Massachusetts, 1979.

    Google Scholar 

  23. C. Chu and J. K. Aggarwal, “Interpretation of Laser Radar Images by a Knowledge-Based System,” in press, to appear in Journal of Machine Vision and Applications.

    Google Scholar 

  24. L. G. Shapiro, “The role of AI in computer vision,” The Second Conference on Artificial Intelligence Applications, Miami Beach, Florida, December 11–13, 1985, pp. 76–81.

    Google Scholar 

  25. T. D. Garvey, “Survey of AI approaches to the integration of information,” Proceedings of SPIE, vol. 782, May 1987, pp. 68–82.

    Article  Google Scholar 

  26. T. J. Fan, G. Medioni, and R. Nevatia, “3D object recognition using surface descriptions,” Proc. DARPA Image Understanding Workshop, April 1988, pp. 383–397.

    Google Scholar 

  27. D. E. Dudgeon, J. G. Verly, and R. L. Delanoy, “An experimental target recognition system for laser radar imagery,” The Proceedings of DARPA Image Understanding Workshop, Palo Alto, California, May 1989, pp. 479–506.

    Google Scholar 

  28. C. Chu and A. Bovik, “Visual surface reconstruction using minimax approximation,” Pattern Recognition, vol. 21, No. 4, 1988, pp. 303–312.

    Article  Google Scholar 

  29. C. Chu, N. Nandhakumar and J. K. Aggarwal, “Image Segmentation Using Laser Radar Data”, Pattern Recognition, vol. 23, no. 6, pp. 569–581, 1990.

    Article  Google Scholar 

  30. C. Chu and J. K. Aggarwal, “The integration of region and edge-based segmentation,” to appear in The Proceedings of the Second International Conference on Computer Vision,Osaka, Japan, December 4–7, 1990.

    Google Scholar 

  31. T. Pavlidis and Y. T. Liow, “Integrating region growing and edge detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-12, no. 3, March 1990, pp. 225–233.

    Google Scholar 

  32. W. J. Clancey, “Heuristic Classification,” Artificial Intelligence,vol. 27, pp. 289350.

    Google Scholar 

  33. B. G. Buchanan and E. H. Shortliffe, Rule-Based Expert Systems, Addison-Wesley, Massachussetts, 1984.

    Google Scholar 

  34. G. Shafer, A Mathematical Theory of Evidence, Princeton University Press, 1976.

    Google Scholar 

  35. H. Asar, “Pyramid based image segmentation using multisensory data,” M. S. Thesis, The University of Texas at Austin, December 1988.

    Google Scholar 

  36. N. Nandhakumar and J.K. Aggarwal, “Integrating Information from Thermal and Visual Images for Scene Analysis”, in Proceedings of SPIE, Vol. 635, 1986, pp. 132–142.

    Google Scholar 

  37. N. Nandhakumar and J.K. Aggarwal, “Multisensor Integration–Experiments in Integrating Thermal and Visual Images for Scene Perception”, The Proceedings of the First International Conference on Computer Vision, London, England, June 8–11, 1987, pp. 83–92.

    Google Scholar 

  38. N. Nandhakumar and J. K. Aggarwal, “Integrated analysis of thermal and visual images for scene interpretation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-10, no. 4, July 1988, pp. 469–481.

    Google Scholar 

  39. N. Nandhakumar and J. K. Aggarwal, “Thermal and visual information fusion for outdoor scene perception,” Proceedings of IEEE Robotics and Automation Conference, Philadelphia, April 25–29, 1988, pp. 1306–1308.

    Google Scholar 

  40. C. Oh, N. Nandhakumar, and J. K. Aggarwal, “Integrated modeling of thermal and visual image generation,” The Proceedings of Computer Vision and Pattern Recognition, San Diego, California, June 4–8, 1989, pp. 356–372.

    Google Scholar 

  41. T. Thepchatri, C.P. Johnson and H. Matlock, “Prediction of Temperature and Stresses in Highway Bridges by A Numerical Procedure Using Daily Weather Reports”, Technical Report 23–1, Center for Highway Research, University of Texas at Austin, 1977.

    Google Scholar 

  42. F.P. Incropera, and D.P. De Witt, Fundamentals of Heat Transfer, John Wiley Sons, Inc., New York, 1981.

    Google Scholar 

  43. W.M. Roshenaw and J.R. Hartnett, Handbook of Heat Transfer, McGraw Hill Book Co., New York, 1973.

    Google Scholar 

  44. M. Kumar, “Use of Reflectivity Ratios to Measure Light Interception by Crops”, Proceedings of SPIE, Vol. 234, 1980, pp. 77–81.

    Article  Google Scholar 

  45. B. V. Karlekar and R. M. Desmond, Engineering Heat Transfer, West Publishing Co., New York, 1977.

    Google Scholar 

  46. C. Ohman, “Practical Methods for Improving Thermal Measurements,” Proceedings of SPIE, Vol. 313, 1981, pp. 204–212.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer and Vision Research Center, University of Texas at Austin, Austin, Texas, 78712, USA

    J. K. Aggarwal & Chen-Chau Chu

Authors
  1. J. K. Aggarwal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chen-Chau Chu
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Computer and Vision Research Center Department of Electrical and Computer Engineering, The University of Texas at Austin, 78712-1084, Austin, TX, USA

    J. K. Aggarwal (Director) (Director)

Rights and permissions

Reprints and permissions

Copyright information

© 1993 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Aggarwal, J.K., Chu, CC. (1993). The Issues, Analysis, and Interpretation of Multi-Sensor Images. In: Aggarwal, J.K. (eds) Multisensor Fusion for Computer Vision. NATO ASI Series, vol 99. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-02957-2_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-02957-2_3

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-08135-4

  • Online ISBN: 978-3-662-02957-2

  • eBook Packages: Springer Book Archive

Publish with us

Policies and ethics

Search

Navigation