Augmented Reality by Integrating Multiple Sensory Modalities for Underwater Scene Understanding

  • Vittorio Murino
  • Andrea Fusiello
Part of the NATO Science Series book series (ASHT, volume 84)


This chapter proposes a method for the integration of acoustic and optical data to enhance the perception of an underwater environment in teleoperation tasks. Off-shore applications are addressed, in which an underwater remotely operated vehicle is approaching an oil rig for inspection, maintenance and repairing tasks. A technique is presented which takes advantage of optical features to segment an acoustic three-dimensional (3-D) image. Cylindrical surfaces are than extracted from 3-D points, and complete cylinders are reconstructed. The final step is to present useful information to the human operator, by displaying the superposition of measured acoustic data and geometric primitives fitted to parts of it, i.e., an augmented reality view. Experimental results with real data are reported showing the effectiveness of the proposed approach.


Augmented Reality Markov Random Field Acoustic Data Complete Subgraph Optical Camera 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    D. H. Ballard and Brown C. M. Computer Vision. Prentice-Hall Inc., 1982.Google Scholar
  2. [2]
    P. Besl and N. McKay. A method for registration of 3-D shapes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2):239–256, February 1992.CrossRefGoogle Scholar
  3. [3]
    R. R. Brooks and S.S. Iyengar. Multi-Sensor Fusion. Prentice Hall, Upper Saddle River, USA, 1998.Google Scholar
  4. [4]
    J. B. Burns, A. R. Hanson, and E. M. Riseman. Extracting straight lines. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8(4):425–456, 1986.CrossRefGoogle Scholar
  5. [5]
    J. F. Canny. A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8(6):679–698, November 1986.CrossRefGoogle Scholar
  6. [6]
    A. Fusiello, R. Giannitrapani, V. Isaia, and V. Murino. Virtual environment modeling by integrated optical and acoustic sensing. In Second International Conference on 3-D Digital Imaging and Modeling (3DIM99), pages 437–446, Ottawa, Canada, 4–8 October 1999. IEEE Computer Society Press.CrossRefGoogle Scholar
  7. [7]
    A. Fusiello and V. Murino. Calibration of an optical/acoustic sensor. In 6th International Conference on Computer Graphics and Image Processing (GKPO2000), 2000. To appear.Google Scholar
  8. [8]
    R. Giannitrapani, A. Trucco, and V. Murino. Segmentation of underwater 3-D acoustical images for augmented and virtual reality applications. In Proceedings of the OCEANS’99 Conference, pages 459–465, Seattle (USA), September 1999. MTS/IEEE.Google Scholar
  9. [9]
    F. Goulette. Automatic CAD modeling of industrial pipes from range images. In International Conference on Recent Advances in 3-D Digital Imaging and Modeling, pages 229–233, May 1997.Google Scholar
  10. [10]
    B. Gunsel, A. K. Jain, and E. Panayirci. Reconstruction and boundary detection of range and intensity images using multiscale MRF representations. CVGIP: Image Understanding, 63(2):353–366, March 1996.Google Scholar
  11. [11]
    R. K. Hansen and P. A. Andersen. A 3-D underwater acoustic camera—properties and applications. In P. Tortoli and L. Masotti, editors, Acoustical Imaging, pages 607–611. Plenum Press, 1996.CrossRefGoogle Scholar
  12. [12]
    M. Hebert, R. Hoffman, A. Johnson, and J. Osborn. Sensor based interior modeling. In American Nuclear Society 6th Topical Meeting on Robotics and Remote Systems (ANS ’95), pages 731–737, February 1995.Google Scholar
  13. [13]
    A. Johnson, P. Leger, R. Hoffman, M. Hebert, and J. Osborn. 3-D object modeling and recognition for telerobotic manipulation. In Proc. IEEE Intelligent Robots and Systems, volume 1, pages 103–110, August 1995.Google Scholar
  14. [14]
    D. Dion Jr. and D. Laurendeau. Generalized cylinders extraction in a range image. In International Conference on Recent Advances in 3-D Digital Imaging and Modeling, pages 141–147, May 1997.Google Scholar
  15. [15]
    D. G. Lowe. Fitting parameterized three-dimensional models to images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13(5):441–450, May 1991.MathSciNetCrossRefGoogle Scholar
  16. [16]
    R. C. Luo and M. G. Kay. Multisensor integration and fusion in intelligent systems. IEEE Transactions on Systems, Man and Cybernetics, 19(5):901–931, September-October 1989.CrossRefGoogle Scholar
  17. [17]
    M. Maimone, L. Matthies, J. Osborn, E. Rollins, J. Teza, and S. Thayer. A photo-realistic 3-D mapping system for extreme nuclear environments: Chornobyl. In Proceedings of the 1998 IEEE/RS J International Conference on Intelligent Robotic Systems (IROS ’98). IEEE, 1998.Google Scholar
  18. [18]
    V. Murino, A. Trucco, and C. Regazzoni. A probabilistic approach to the coupled reconstruction and restoration of underwater acoustic images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(l):9–22, January 1998.CrossRefGoogle Scholar
  19. [19]
    P. Perona and J. Malik. Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12(7):629–639, 1990.CrossRefGoogle Scholar
  20. [20]
    L. Robert. Camera calibration without feature extraction. Computer Vision, Graphics, and Image Processing, 63(2):314–325, March 1996.CrossRefGoogle Scholar
  21. [21]
    P. J. Rousseeuw and A. M. Leroy. Robust regression & outlier detection. Wiley, 1987.zbMATHCrossRefGoogle Scholar
  22. [22]
    G. Scott and H. Longuet-Higgins. An algorithm for associating the features of two images. In Proceedings of the Royal Society of London B, volume 244, pages 21–26, 1991.Google Scholar
  23. [23]
    C. Yu W. Lie and Y. Chen. Model-based recognition and positioning of polyhedra using intensity-guided range sensing and interpretation in 3-D space. Pattern Recognition, 23:983–997, 1990.CrossRefGoogle Scholar
  24. [24]
    G. H. Zhang and A. Wallace. Physical modeling and combination of range and intensity edge data. CVGIP: Image Understanding, 58(2):191–220, September 1993.CrossRefGoogle Scholar
  25. [25]
    Z. Zhang, R. Deriche, O. Faugeras, and Q.-T. Luong. A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry. Artificial Intelligence, 78(l-2):87–119, 1995.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2000

Authors and Affiliations

  • Vittorio Murino
  • Andrea Fusiello

There are no affiliations available

Personalised recommendations