Skip to main content
SpringerLink
Log in
Book cover

3D Imaging, Analysis and Applications pp 35–94Cite as

Passive 3D Imaging

  • Chapter

Abstract

We describe passive, multiple-view 3D imaging systems that recover 3D information from scenes that are illuminated only with ambient lighting. Much of the material is concerned with using the geometry of stereo 3D imaging to formulate estimation problems. Firstly, we present an overview of the common techniques used to recover 3D information from camera images. Secondly, we discuss camera modeling and camera calibration as an essential introduction to the geometry of the imaging process and the estimation of geometric parameters. Thirdly, we focus on 3D recovery from multiple views, which can be obtained using multiple cameras at the same time (stereo), or a single moving camera at different times (structure from motion). Epipolar geometry and finding image correspondences associated with the same 3D scene point are two key aspects for such systems, since epipolar geometry establishes the relationship between two camera views, while depth information can be inferred from the correspondences. The details of both stereo and structure from motion, the two essential forms of multiple-view 3D reconstruction technique, are presented. Towards the end of the chapter, we present several real-world applications.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   89.95
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    http://www.videredesign.com.

  2. 2.

    You may wish to compare l T x=0 to two well-known parameterizations of a line in the (x,y) plane, namely: ax+by+c=0 and y=mx+c and, in each case, write down homogeneous coordinates for the point x and the line l.

  3. 3.

    We use a tilde to differentiate n-tuple inhomogeneous coordinates from (n+1)-tuple homogeneous coordinates.

  4. 4.

    We need to use a variety of image coordinate normalizations in this chapter. For simplicity, we will use the same subscript n, but it will be clear about how the normalization is achieved.

  5. 5.

    Skew models a lack of orthogonality between the two image sensor sampling directions. For most imaging situations it is zero.

  6. 6.

    The same homogeneous image coordinates up to scale or the same inhomogeneous image coordinates.

  7. 7.

    Due to the scale equivalence of homogeneous coordinates.

  8. 8.

    No three points collinear.

  9. 9.

    ‘Approximately’, because of noise in the imaged corner positions supplied to the calibration process.

  10. 10.

    Extrinsic parameters are always not known in a structure from motion problem, they are part of what we are trying to solve for. Intrinsic parameters may or may not be known, depending on the application.

  11. 11.

    The length of the baseline is the magnitude of the extrinsic translation vector, t.

  12. 12.

    There are several other approaches, such as the seven-point algorithm.

  13. 13.

    An inlier is a putative correspondence that lies within some threshold of its expected position predicted by F. In other words image points must lie within a threshold from their epipolar lines generated by F.

  14. 14.

    Bundle adjustment methods appeared several decades ago in the photogrammetry literature and are now used widely in the computer vision community.

  15. 15.

    http://www.ptgrey.com/products/stereo.asp.

  16. 16.

    http://www.tyzx.com/products/DeepSeaG2.html.

References

  1. Barfoot, T., Se, S., Jasiobedzki, P.: Vision-based localization and terrain modelling for planetary rovers. In: Howard, A., Tunstel, E. (eds.) Intelligence for Space Robotics, pp. 71–92. TSI Press, Albuquerque (2006)

    Google Scholar 

  2. Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: SURF: speeded up robust features. Comput. Vis. Image Underst. 110(3), 346–359 (2008)

    Article  Google Scholar 

  3. Belhumeur, P.N.: A Bayesian approach to binocular stereopsis. Int. J. Comput. Vis. 19(3), 237–260 (1996)

    Article  Google Scholar 

  4. Bergen, J.R., Anandan, P., Hanna, K.J., Hinogorani, R.: Hierarchical model-based motion estimation. In: European Conference on Computer Vision (ECCV), Italy, pp. 237–252 (1992)

    Google Scholar 

  5. Birchfield, S., Tomasi, C.: Depth discontinuities by pixel-to-pixel stereo. Int. J. Comput. Vis. 35(3), 269–293 (1999)

    Article  Google Scholar 

  6. Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)

    Article  Google Scholar 

  7. Brown, D.C.: Decentering distortion of lenses. Photogramm. Eng. 32(3), 444–462 (1966)

    Google Scholar 

  8. Brown, D.C.: Close-range camera calibration. Photogramm. Eng. 37(8), 855–866 (1971)

    Google Scholar 

  9. Camera Calibration Toolbox for MATLAB: http://www.vision.caltech.edu/bouguetj/calib_doc/. Accessed 20th October 2011

  10. Chen, Z., Pears, N.E., Liang, B.: Monocular obstacle detection using Reciprocal-Polar rectification. Image Vis. Comput. 24(12), 1301–1312 (2006)

    Article  Google Scholar 

  11. Cox, I.J., Hingorani, S.L., Rao, S.B., Maggs, B.M.: A maximum likelihood stereo algorithm. Comput. Vis. Image Underst. 63(3), 542–567 (1996)

    Article  Google Scholar 

  12. Coxeter, H.S.M.: Projective Geometry, 2nd edn. Springer, Berlin (2003)

    MATH  Google Scholar 

  13. Criminisi, A., Reid, I., Zisserman, A.: Single view metrology. Int. J. Comput. Vis. 40(2), 123–148 (2000)

    Article  MATH  Google Scholar 

  14. Davison, A., Reid, I., Molton, N., Stasse, O.: MonoSLAM: real-time single camera SLAM. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 1052–1067 (2007)

    Article  Google Scholar 

  15. Faugeras, O., Luong, Q.T.: The Geometry of Multiple Images. MIT Press, Cambridge (2001)

    MATH  Google Scholar 

  16. Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    Article  MathSciNet  Google Scholar 

  17. Garding, J.: Shape from texture for smooth curved surfaces in perspective projection. J. Math. Imaging Vis. 2, 329–352 (1992)

    Article  Google Scholar 

  18. Harris, C., Stephens, M.J.: A combined corner and edge detector. In: Alvey Vision Conference, pp. 147–152 (1988)

    Google Scholar 

  19. Harley, R.I.: In defense of the 8-point algorithm. IEEE Trans. Pattern Anal. Mach. Intell. 19(6), 580–593 (1997)

    Article  Google Scholar 

  20. Hartley, R.I.: Theory and practice of projective rectification. Int. J. Comput. Vis. 35(2), 115–127 (1999)

    Article  Google Scholar 

  21. Hartley, R.I., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2004)

    Book  MATH  Google Scholar 

  22. Horn, B.K.P., Brooks, M.J. (eds.): Shape from Shading. MIT Press, Cambridge (1989)

    Google Scholar 

  23. Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artif. Intell. 17, 185–203 (1981)

    Article  Google Scholar 

  24. Huang, R., P, S.W.A.: A shape-from-shading framework for satisfying data-closeness and structure-preserving smoothness constraints. In: Proceedings of the British Machine Vision Conference (2009)

    Google Scholar 

  25. Kolmogorov, V., Zabih, R.: Computing visual correspondence with occlusions using graph cuts. In: International Conference on Computer Vision (ICCV), Vancouver, pp. 508–515 (2001)

    Google Scholar 

  26. Konolige, K.: Small vision system: hardware and implementation. In: Proc. Int. Symp. on Robotics Research, Hayama, Japan, pp. 111–116 (1997)

    Google Scholar 

  27. Longuet-Higgins, H.C.: A computer algorithm for re-constructing a scene from two projections. Nature 293, 133–135 (1981)

    Article  Google Scholar 

  28. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  29. Lucas, B.D., Kanade, T.: An interactive image registration technique with an application in stereo vision. In: International Joint Conference on Artificial Intelligence (IJCAI), Vancouver, pp. 674–679 (1981)

    Google Scholar 

  30. Ma, Y., Soatto, S., Kosecka, J., Sastry, S.S.: An Invitation to 3-D Vision. Springer, New York (2003)

    Google Scholar 

  31. Maimone, M., Biesiadecki, J., Tunstel, E., Cheng, Y., Leger, C.: Surface navigation and mobility intelligence on the Mars Exploration Rovers. In: Howard, A., Tunstel, E. (eds.) Intelligence for Space Robotics, pp. 45–69. TSI Press, Albuquerque (2006)

    Google Scholar 

  32. Maimone, M., Cheng, Y., Matthies, L.: Two years of visual odometry on the mars exploration rovers. J. Field Robot. 24(3), 169–186 (2007)

    Article  Google Scholar 

  33. Mallon, J., Whelan, P.F.: Projective rectification from the fundamental matrix. In: Image and Vision Computing, pp. 643–650 (2005)

    Google Scholar 

  34. The Middlebury stereo vision page: http://vision.middlebury.edu/stereo/. Accessed 16th November 2011

  35. Moons, T., Van Gool, L., Vergauwen, M.: 3D reconstruction from multiple images. Found. Trends Comput. Graph. Vis. 4(4), 287–404 (2010)

    Article  Google Scholar 

  36. Mordohai, P., et al.: Real-time video-based reconstruction of urban environments. In: International Workshop on 3D Virtual Reconstruction and Visualization of Complex Architectures (3D-ARCH), Zurich, Switzerland (2007)

    Google Scholar 

  37. Nayar, S.K., Nakagawa, Y.: Shape from focus. IEEE Trans. Pattern Anal. Mach. Intell. 16(8), 824–831 (1994)

    Article  Google Scholar 

  38. Nister, D.: Automatic passive recovery of 3D from images and video. In: International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT), Thessaloniki, Greece, pp. 438–445 (2004)

    Chapter  Google Scholar 

  39. Nister, D.: An efficient solution to the five-point relative pose problem. IEEE Trans. Pattern Anal. Mach. Intell. 26(6), 756–770 (2004)

    Article  Google Scholar 

  40. Open source computer vision library: http://opencv.willowgarage.com/wiki/. Accessed 20th October 2011

  41. Pentland, A.P.: A new sense for depth of field. IEEE Trans. Pattern Anal. Mach. Intell. 9(4), 523–531 (1987)

    Article  Google Scholar 

  42. Pollefeys, M., Koch, R., Van Gool, L.: A simple and efficient rectification method for general motion. In: International Conference on Compute Vision (ICCV), Kerkyra, Greece, pp. 496–501 (1999)

    Google Scholar 

  43. Pollefeys, M., Van Gool, L., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., Koch, R.: Visual modeling with a hand-held camera. Int. J. Comput. Vis. 59(3), 207–232 (2004)

    Article  Google Scholar 

  44. Quam, L.H.: Hierarchical warp stereo. In: Image Understanding Workshop, New Orleans, pp. 149–155 (1984)

    Google Scholar 

  45. Roy, S., Cox, I.J.: A maximum-flow formulation of the N-camera stereo correspondence problem. In: International Conference on Computer Vision (ICCV), Bombay, pp. 492–499 (1998)

    Google Scholar 

  46. Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vis. 47(1/2/3), 7–42 (2002)

    Article  MATH  Google Scholar 

  47. Se, S., Jasiobedzki, P.: Photo-realistic 3D model reconstruction. In: IEEE International Conference on Robotics and Automation, Orlando, Florida, pp. 3076–3082 (2006)

    Google Scholar 

  48. Se, S., Jasiobedzki, P.: Stereo-vision based 3D modeling and localization for unmanned vehicles. Int. J. Intell. Control Syst. 13(1), 47–58 (2008)

    Google Scholar 

  49. Se, S., Lowe, D., Little, J.: Mobile robot localization and mapping with uncertainty using scale-invariant visual landmarks. Int. J. Robot. Res. 21(8), 735–758 (2002)

    Article  Google Scholar 

  50. Se, S., Firoozfam, P., Goldstein, N., Dutkiewicz, M., Pace, P.: Automated UAV-based video exploitation for mapping and surveillance. In: International Society for Photogrammetry and Remote Sensing (ISPRS) Commission I Symposium, Calgary (2010)

    Google Scholar 

  51. Seitz, S., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), New York, pp. 519–526 (2006)

    Google Scholar 

  52. Smith, S.M., Brady, J.M.: SUSAN—a new approach to low level image processing. Int. J. Comput. Vis. 23(1), 45–78 (1997)

    Article  Google Scholar 

  53. Sun, J., Zheng, N., Shum, H.: Stereo matching using belief propagation. IEEE Trans. Pattern Anal. Mach. Intell. 25(7), 787–800 (2003)

    Article  Google Scholar 

  54. Tappen, M.F., Freeman, W.T.: Comparison of graph cuts with belief propagation for stereo, using identical MRF parameters. In: International Conference on Computer Vision (ICCV), Nice, France, pp. 900–907 (2003)

    Chapter  Google Scholar 

  55. Torr, P.H.S., Murray, D.: The development and comparison of robust methods for estimating the fundamental matrix. Int. J. Comput. Vis. 24(3), 271–300 (1997)

    Article  Google Scholar 

  56. Torresani, L., Hertzmann, A., Bregler, C.: Non-rigid structure-from-motion: estimating shape and motion with hierarchical priors. IEEE Trans. Pattern Anal. Mach. Intell. 30(5), 878–892 (2008)

    Article  Google Scholar 

  57. Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzigibbon, A.W.: Bundle adjustment—a modern synthesis. In: International Workshop on Vision Algorithms, Kerkyra, Greece, pp. 298–372 (1999)

    Google Scholar 

  58. Tsai, R.Y.: A versatile camera calibration technique for high-accuracy 3D machine vision metrology using off-the-shelf TV cameras and lenses. IEEE J. Robot. Autom. 3(4), 323–344 (1987)

    Article  Google Scholar 

  59. Videre Design: http://www.videredesign.com/. Accessed 16th November 2011

  60. White, R., Forsyth, D.A.: Combining cues: shape from shading and texture. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), New York, pp. 1809–1816 (2006)

    Google Scholar 

  61. Woodham, R.J.: Analysing images of curved surfaces. Artif. Intell. 17, 117–140 (1981)

    Article  Google Scholar 

  62. Yamauchi, B.: Autonomous urban reconnaissance using man-portable UGVs. In: Unmanned Systems Technology. Orlando, Florida, SPIE, vol. 6230 (2006)

    Google Scholar 

  63. Yang, R., Pollefeys, M.: A versatile stereo implementation on commodity graphics hardware. Real-Time Imaging 11(1), 7–18 (2005)

    Article  Google Scholar 

  64. Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)

    Article  Google Scholar 

  65. Zhang, R., Tsai, P.S., Cryer, J.E., Shah, M.: Shaping from shading: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 21(8), 690–706 (1999)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. MDA Systems Ltd., 13800 Commerce Parkway, Richmond, BC, V6V 2J3, Canada

    Stephen Se

  2. Department of Computer Science, University of York, Deramore Lane, York, YO10 5GH, UK

    Nick Pears

Authors
  1. Stephen Se
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nick Pears
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stephen Se .

Editor information

Editors and Affiliations

  1. Department of Computer Science, University of York, Deramore Lane, Heslington, York, YO10 5GH, United Kingdom

    Nick Pears

  2. Department of Computer Science, Aberystwyth University, Llandinam Building, Ceredigion, Aberystwyth, SY23 3DB, United Kingdom

    Yonghuai Liu

  3. Institute of Geography and Earth Science, Aberystwyth University, Penglais Campus, Ceredigion, Aberystwyth, SY23 3DB, United Kingdom

    Peter Bunting

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag London

About this chapter

Cite this chapter

Se, S., Pears, N. (2012). Passive 3D Imaging. In: Pears, N., Liu, Y., Bunting, P. (eds) 3D Imaging, Analysis and Applications. Springer, London. https://doi.org/10.1007/978-1-4471-4063-4_2

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-4063-4_2

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-4062-7

  • Online ISBN: 978-1-4471-4063-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation