Advertisement

The Issues, Analysis, and Interpretation of Multi-Sensor Images

  • J. K. Aggarwal
  • Chen-Chau Chu
Conference paper
Part of the NATO ASI Series book series (volume 99)

Abstract

Past research in computer vision has shown that image interpretation is a highly underconstrained task. Information fusion from multiple cues from the same image and from multiple views using the same modality have been marginally successful. Recently the fusion of information from different modalities of sensing has been studied to further constrain the interpretation. This paper presents an overview of approaches developed for image segmentation and analysis using multi-sensor fusion. We present examples of three systems using different modalities. These examples include a system for image segmentation and interpretation using ladar (laser radar) and thermal images, a system using registered thermal and visual images for surface heat flux analysis, and an image synthesis system that generates visual and thermal images based on the internal heat flow in objects.

Keywords

Image Segmentation Surface Heat Flux Thermal Image Object Surface Laser Radar 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. [1]
    N. Nandhakumar, “Application of artificial Intelligence to multisensory vision,” invited talk at IEEE Computer Society Workshop on Artificial Intelligence for Computer Vision, San Diego, June 5, 1989.Google Scholar
  2. [2]
    B. K. P. Horn, “Robot Vision,” MIT Press, Cambridge, Massachusetts, 1986.Google Scholar
  3. [3]
    M. L. Moerdler and J. R. Kender, “An approach to the fusion of multiple shape from texture algorithms,” The Proceedings of IEEE Workshop on Spatial Reasoning and Multi-Sensor Fusion, 1987, pp. 272–281.Google Scholar
  4. [4]
    W. E. L. Crimson, From Images to Surfaces: A Computational Study of the Human Early Vision System, MIT Press, Cambridge, Massachusetts, 1981.Google Scholar
  5. [5]
    U. Dhond and J. K. Aggarwal, “Structure from stereo: a review,” IEEE Transactions on System, Man, and Cybernetics, SMC-19, no. 6, 1989, pp. 1489–1510.Google Scholar
  6. [6]
    J. K. Aggarwal and N. Nandhakumar, “On the computation of motion from sequences of images — a review,” Proceedings of the IEEE, vol. 76, no. 8, August 1988, pp. 917–935.CrossRefGoogle Scholar
  7. [7]
    J. K. Aggarwal and Y. F. Wang, “Analysis of a sequence of images using point and line correspondences,” The Proceedings of IEEE Robotics and Automation Conference, March 31-April 3, 1987, pp. 1275–1280.Google Scholar
  8. [8]
    C. W. Tong, S. K. Rogers, J. P. Mills, and M. K. Kabrisky, “Multisensor data fusion of laser radar and forward looking infrared (FLIR) for target segmentation and enhancement,” Proceedings of SPIE, vol. 782, pp. 10–19.Google Scholar
  9. [9]
    P. Allen, “A framework for implementing multi-sensor robotic tasks,” The Proceedings of DARPA Image Understanding Workshop, 1987, pp. 392–398.Google Scholar
  10. [10]
    A. K. Jain and R. Hoffman, “Evidence-based recognition of 3-D objects,” IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-10, no. 6, November 1988, pp. 783–802.Google Scholar
  11. [11]
    Y. F. Wang and J. K. Aggarwal, “On modeling 3D objects using multiple sensory data,” The Proceedings of IEEE Robotics and Automation Conference, March 31-April 3, 1987, pp. 1098–1103.Google Scholar
  12. [12]
    Y. F. Wang, A. Mitiche, and J. K. Aggarwal, “Computation of surface orientation and structure of objects using grid coding,” IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-9, no. 1, January 1987, pp. 129–137.Google Scholar
  13. [13]
    C. H. Chien and J. K. Aggarwal, “Volume surface Octree for the Representation of 3-D Objects,” Computer Vision, Graphics, and Image Processing, vol. 36, 1986, pp. 100–113.Google Scholar
  14. [14]
    C. H. Chien and J. K. Aggarwal, “Identification of 3D objects from multiple silhouettes using quadtree octree,” Computer Vision, Graphics, and Image Processing, vol. 36, 1986, pp. 256–273.Google Scholar
  15. [15]
    C. H. Chien and J. K. Aggarwal, “Reconstruction and recognition of 3D objects from occluding contours and silhouettes,” Ph.D. Dissertation, University of Texas at Austin, May 1987.Google Scholar
  16. [16]
    C. H. Chien and J. K. Aggarwal, “Computation of Volume surface Octree from Contours and Silhouettes of Multiple Views,” Proc. Comp. Vision and Pattern Recog. Conf., Miami Beach, FL, 1986, pp. 220–225.Google Scholar
  17. [17]
    C. H. Chien, Y. B. Sim and J. K. Aggarwal, “Generation of Volume Surface Octree from Range Data,” Proc. Comp. Vision and Pattern Recog. Conf., Ann Arbor, MI, 1988, pp. 254–260.Google Scholar
  18. [18]
    B. Gil, A. Mitiche, and J. K. Aggarwal, “Experiments in combining intensity and range edge maps,” Computer Vision, Graphics, and Image Processingvol. 21, 1983, pp. 395–411.Google Scholar
  19. [19]
    M. J. Magee, B. A. Boyter, C. Chien, and J. K. Aggarwal, “Experiments in intensity guided range sensing recognition of three-dimensional objects,” IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-7, no. 6, November 1985, pp. 629–637.Google Scholar
  20. [20]
    J. K. Aggarwal and N. Nandhakumar, “Multisensor fusion for automatic scene interpretation - research issues and directions,” Proceedings NSF Workshop on Range Image Processing, edit. R. Jain, March 1988.Google Scholar
  21. [21]
    P. J. Besl and R. C. Jain, “Segmentation Through Variable-Order Surface Fitting,” IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-10, no. 2, March 1988, pp. 167–192.Google Scholar
  22. [22]
    C. G. Bachman, Laser Radar Systems and Techniques, Artech House, Dedham Massachusetts, 1979.Google Scholar
  23. [23]
    C. Chu and J. K. Aggarwal, “Interpretation of Laser Radar Images by a Knowledge-Based System,” in press, to appear in Journal of Machine Vision and Applications.Google Scholar
  24. [24]
    L. G. Shapiro, “The role of AI in computer vision,” The Second Conference on Artificial Intelligence Applications, Miami Beach, Florida, December 11–13, 1985, pp. 76–81.Google Scholar
  25. [25]
    T. D. Garvey, “Survey of AI approaches to the integration of information,” Proceedings of SPIE, vol. 782, May 1987, pp. 68–82.CrossRefGoogle Scholar
  26. [26]
    T. J. Fan, G. Medioni, and R. Nevatia, “3D object recognition using surface descriptions,” Proc. DARPA Image Understanding Workshop, April 1988, pp. 383–397.Google Scholar
  27. [27]
    D. E. Dudgeon, J. G. Verly, and R. L. Delanoy, “An experimental target recognition system for laser radar imagery,” The Proceedings of DARPA Image Understanding Workshop, Palo Alto, California, May 1989, pp. 479–506.Google Scholar
  28. [28]
    C. Chu and A. Bovik, “Visual surface reconstruction using minimax approximation,” Pattern Recognition, vol. 21, No. 4, 1988, pp. 303–312.CrossRefGoogle Scholar
  29. [29]
    C. Chu, N. Nandhakumar and J. K. Aggarwal, “Image Segmentation Using Laser Radar Data”, Pattern Recognition, vol. 23, no. 6, pp. 569–581, 1990.CrossRefGoogle Scholar
  30. [30]
    C. Chu and J. K. Aggarwal, “The integration of region and edge-based segmentation,” to appear in The Proceedings of the Second International Conference on Computer Vision,Osaka, Japan, December 4–7, 1990.Google Scholar
  31. [31]
    T. Pavlidis and Y. T. Liow, “Integrating region growing and edge detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. PAMI-12, no. 3, March 1990, pp. 225–233.Google Scholar
  32. [32]
    W. J. Clancey, “Heuristic Classification,” Artificial Intelligence,vol. 27, pp. 289350.Google Scholar
  33. [33]
    B. G. Buchanan and E. H. Shortliffe, Rule-Based Expert Systems, Addison-Wesley, Massachussetts, 1984.Google Scholar
  34. [34]
    G. Shafer, A Mathematical Theory of Evidence, Princeton University Press, 1976.Google Scholar
  35. [35]
    H. Asar, “Pyramid based image segmentation using multisensory data,” M. S. Thesis, The University of Texas at Austin, December 1988.Google Scholar
  36. [36]
    N. Nandhakumar and J.K. Aggarwal, “Integrating Information from Thermal and Visual Images for Scene Analysis”, in Proceedings of SPIE, Vol. 635, 1986, pp. 132–142.Google Scholar
  37. [37]
    N. Nandhakumar and J.K. Aggarwal, “Multisensor Integration–Experiments in Integrating Thermal and Visual Images for Scene Perception”, The Proceedings of the First International Conference on Computer Vision, London, England, June 8–11, 1987, pp. 83–92.Google Scholar
  38. [38]
    N. Nandhakumar and J. K. Aggarwal, “Integrated analysis of thermal and visual images for scene interpretation,” IEEE Transactions on Pattern Analysis and Machine Intelligence, PAMI-10, no. 4, July 1988, pp. 469–481.Google Scholar
  39. [39]
    N. Nandhakumar and J. K. Aggarwal, “Thermal and visual information fusion for outdoor scene perception,” Proceedings of IEEE Robotics and Automation Conference, Philadelphia, April 25–29, 1988, pp. 1306–1308.Google Scholar
  40. [40]
    C. Oh, N. Nandhakumar, and J. K. Aggarwal, “Integrated modeling of thermal and visual image generation,” The Proceedings of Computer Vision and Pattern Recognition, San Diego, California, June 4–8, 1989, pp. 356–372.Google Scholar
  41. [41]
    T. Thepchatri, C.P. Johnson and H. Matlock, “Prediction of Temperature and Stresses in Highway Bridges by A Numerical Procedure Using Daily Weather Reports”, Technical Report 23–1, Center for Highway Research, University of Texas at Austin, 1977.Google Scholar
  42. [42]
    F.P. Incropera, and D.P. De Witt, Fundamentals of Heat Transfer, John Wiley Sons, Inc., New York, 1981.Google Scholar
  43. [43]
    W.M. Roshenaw and J.R. Hartnett, Handbook of Heat Transfer, McGraw Hill Book Co., New York, 1973.Google Scholar
  44. [44]
    M. Kumar, “Use of Reflectivity Ratios to Measure Light Interception by Crops”, Proceedings of SPIE, Vol. 234, 1980, pp. 77–81.CrossRefGoogle Scholar
  45. [45]
    B. V. Karlekar and R. M. Desmond, Engineering Heat Transfer, West Publishing Co., New York, 1977.Google Scholar
  46. [46]
    C. Ohman, “Practical Methods for Improving Thermal Measurements,” Proceedings of SPIE, Vol. 313, 1981, pp. 204–212.CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1993

Authors and Affiliations

  • J. K. Aggarwal
    • 1
  • Chen-Chau Chu
    • 1
  1. 1.Computer and Vision Research CenterUniversity of Texas at AustinAustinUSA

Personalised recommendations