Foundations and Applications of 3D Imaging

Part of the KAIST Research Series book series (KAISTRS)


Two-dimensional imaging through digital photography has been a main application of mobile computing devices, such as smart phones, during the last decade. Expanding the dimensions of digital imaging, the recent advances in 3D imaging technology are about to be combined with such smart devices, resulting in broadened applications of 3D imaging. This chapter presents the foundations of 3D imaging, that is, the relationship between disparity and depth in a stereo camera system, and it surveys a general workflow to build a 3D model from sensor data. In addition, recent advanced 3D imaging applications are introduced: hyperspectral 3D imaging, multispectral photometric stereo and stereo fusion of refractive and binocular stereo.


Stereo imaging Hyperspectral 3D imaging 



This work was supported by a Korea NRF grant (2013R1A1A1010165) and the Center for Integrated Smart Sensors, funded by the Ministry of Science, ICT & Future Planning, as the Global Frontier Project.


  1. 1.
    Baek SH, Kim MH (2014) Stereo fusion using a refractive medium on a binocular base. In: Proceedings Asian conference on computer vision (ACCV 2014). Springer, LNCS, Singapore, pp 1–16Google Scholar
  2. 2.
    Baek SH, Kim MH (2015) Stereo fusion: combining refractive and binocular disparity. In: Computer vision and image understanding (CVIU), pp 1–42Google Scholar
  3. 3.
    Bando Y, Chen BY, Nishita T (2008) Extracting depth and matte using a color-filtered aperture. ACM Trans Graph 27(5):134:1–134:9Google Scholar
  4. 4.
    Barsky S, Petrou M (2003) The 4-source photometric stereo technique for three-dimensional surfaces in the presence of highlights and shadows. IEEE Trans Pattern Anal Mach Intell 25(10):1239–1252CrossRefGoogle Scholar
  5. 5.
    Basri R, Jacobs DW, Kemelmacher I (2007) Photometric stereo with general, unknown lighting. Int J Comput Vision 72(3):239–257Google Scholar
  6. 6.
    Bernardini F, Rushmeier H (2002) The 3D model acquisition pipeline. Comput Graph Forum 21(2):149CrossRefGoogle Scholar
  7. 7.
    Bouguet JY, Perona P (1998) 3D photography on your desk. In: ICCV, pp 43–52Google Scholar
  8. 8.
    Boykov Y, Veksler O, Zabih R (2001) Fast approximate energy minimization via graph cuts. IEEE Trans Pattern Anal Mach Intell 23(11):1222–1239CrossRefGoogle Scholar
  9. 9.
    Brauers J, Schulte N, Aach T (2007) Modeling and compensation of geometric distortions of multispectral cameras with optical bandpass filter wheels. In: 15th European signal processing conference, vol 15, pp 1902–1906Google Scholar
  10. 10.
    Chandraker M, Agarwal S, Kriegman D (2007) Shadowcuts: photometric stereo with shadows. In: IEEE conference on computer vision and pattern recognition, CVPR’07. IEEE, pp 1–8Google Scholar
  11. 11.
    Chen Z, Wong K, Matsushita Y, Zhu X, Liu M (2011) Self-calibrating depth from refraction. In: Proceedings international conference on computer vision (ICCV), pp 635–642Google Scholar
  12. 12.
    Chen Z, Wong KYK, Matsushita Y, Zhu X (2013) Depth from refraction using a transparent medium with unknown pose and refractive index. Int J Comput Vision 8:1–15Google Scholar
  13. 13.
    Farouk M, Rifai IE, Tayar SE, Shishiny HE, Hosny M, Rayes ME, Gomes J, Giordano F, Rushmeier HE, Bernardini F, Magerlein K (2003) Scanning and processing 3D objects for web display. In: Proceedings international conference on 3D digital imaging and modeling (3DIM), pp 310–317Google Scholar
  14. 14.
    Furukawa Y, Ponce J (2010) Accurate, dense, and robust multiview stereopsis. IEEE Trans Pattern Anal Mach Intell 32(8):1362–1376CrossRefzbMATHGoogle Scholar
  15. 15.
    Gallup D, Frahm JM, Mordohai P, Pollefeys M (2008) Variable baseline/resolution stereo. In: Proceedings on computer vision and pattern recognition (CVPR), pp 1–8Google Scholar
  16. 16.
    Gao C, Ahuja N (2004) Single camera stereo using planar parallel plate. In: Proceedings international conference on pattern recognition (ICPR), vol 4, pp 108–111Google Scholar
  17. 17.
    Gao C, Ahuja N (2006) A refractive camera for acquiring stereo and super-resolution images. In: Proceedings on computer vision and pattern recognition (CVPR), pp 2316–2323Google Scholar
  18. 18.
    Gupta M, Nayar SK (2012) Micro phase shifting. In: Proceedings on computer vision and pattern recognition (CVPR), pp 813–820Google Scholar
  19. 19.
    He K, Sun J, Tang X (2010) Guided image filtering. In: Proceedings on European conference on computer vision (ECCV). Springer, pp 1–14Google Scholar
  20. 20.
    Hecht E (1987) Optics. Addison-Wesley, ReadingGoogle Scholar
  21. 21.
    Hernández C, Vogiatzis G, Cipolla R (2008) Shadows in three-source photometric stereo. In: Computer vision–ECCV 2008. Springer, pp 290–303Google Scholar
  22. 22.
    Holroyd M, Lawrence J, Zickler T (2010) A coaxial optical scanner for synchronous acquisition of 3D geometry and surface reflectance. ACM Trans Graph (Proc SIGGRAPH 2010) 29(3):99:1–99:12Google Scholar
  23. 23.
    Kajiya JT (1986) The rendering equation. In: Proc. ACM SIGGRAPH Computer Graphics ‘86, vol 20, pp 143–150Google Scholar
  24. 24.
    Kim MH, Harvey TA, Kittle DS, Rushmeier H, Dorsey J, Prum RO, Brady DJ (2012) 3D imaging spectroscopy for measuring hyperspectral patterns on solid objects. ACM Trans Graph (Proc SIGGRAPH 2014) 31(4):38:1–38:11Google Scholar
  25. 25.
    Lanman D, Taubin G (2009) Build your own 3D scanner. ACM SIGGRAPH 2009 Courses on—SIGGRAPH 2009. ACM Press, New York, pp 1–94Google Scholar
  26. 26.
    Lee D, Kweon I (2000) A novel stereo camera system by a biprism. IEEE Trans Rob Autom 16(5):528–541CrossRefGoogle Scholar
  27. 27.
    Levin A, Fergus R, Durand F, Freeman WT (2007) Image and depth from a conventional camera with a coded aperture. ACM Trans Graphics 26(3):70:1–70:9Google Scholar
  28. 28.
    Liao M, Huang X, Yang R (2011) Interreflection removal for photometric stereo by using spectrum-dependent albedo. In: Proceedings on computer vision and pattern recognition (CVPR), pp 689–696Google Scholar
  29. 29.
    Liu C, Yuen J, Torralba A (2011) Sift flow: dense correspondence across scenes and its applications. IEEE Trans Pattern Anal Mach Intell 33(5):978–994CrossRefGoogle Scholar
  30. 30.
    Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vision 60(2):91–110CrossRefGoogle Scholar
  31. 31.
    Mansouri A, Lathuiliere A, Marzani FS, Voisin Y, Gouton P (2007) Toward a 3d multispectral scanner: an application to multimedia. IEEE MultiMedia 14(1):40–47CrossRefGoogle Scholar
  32. 32.
    Nakabo Y, Mukai T, Hattori Y, Takeuchi Y, Ohnishi N (2005) Variable baseline stereo tracking vision system using high-speed linear slider. In: Proceedings international conference on robotics and automation (ICRA), pp 1567–1572Google Scholar
  33. 33.
    Nam G, Kim MH (2014) Multispectral photometric stereo for acquiring high-fidelity surface normals. IEEE Comput Graphics Appl 34(6):57–68MathSciNetCrossRefGoogle Scholar
  34. 34.
    Nayar SK, Ikeuchi K, Kanade T (1991) Shape from interreflections. Int J Comput Vision 6(3):173–195CrossRefGoogle Scholar
  35. 35.
    Nayar SK, Krishnan G, Grossberg MD, Raskar R (2006) Fast separation of direct and global components of a scene using high frequency illumination. ACM Trans Graph 25(3):935–944CrossRefGoogle Scholar
  36. 36.
    Nishimoto Y, Shirai Y (1987) A feature-based stereo model using small disparities. In: Proceedings on computer vision and pattern recognition (CVPR), pp 192–196Google Scholar
  37. 37.
    Okutomi M, Kanade T (1993) A multiple-baseline stereo. IEEE Trans Pattern Anal Mach Intell 15(4):353–363CrossRefGoogle Scholar
  38. 38.
    Seitz SM, Curless B, Diebel J, Scharstein D, Szeliski R (2006) A comparison and evaluation of multi-view stereo reconstruction algorithms. In: Proceedings on computer vision and pattern recognition (CVPR), pp 519–528Google Scholar
  39. 39.
    Shimizu M, Okutomi M (2006) Reflection stereo-novel monocular stereo using a transparent plate. In: Proceedings Canadian conference on computer and robot vision (CRV). IEEE, pp 14–14Google Scholar
  40. 40.
    Shimizu M, Okutomi M (2007) Monocular range estimation through a double-sided half-mirror plate. In: Proceedings Canadian conference on computer and robot vision (CRV). IEEE, pp 347–354Google Scholar
  41. 41.
    Sun J, Smith M, Smith L, Midha S, Bamber J (2007) Object surface recovery using a multi-light photometric stereo technique for non-lambertian surfaces subject to shadows and specularities. Image Vis Comput 25(7):1050–1057CrossRefGoogle Scholar
  42. 42.
    Takatani T, Matsushita Y, Lin S, Mukaigawa Y, Yagi Y (2013) Enhanced photometric stereo with multispectral images. In: International conference on machine vision applications (MVA). IAPR. pp 1–4Google Scholar
  43. 43.
    Verbiest F, Van Gool L (2008) Photometric stereo with coherent outlier handling and confidence estimation. In: Proceedings on computer vision and pattern recognition (CVPR), pp 1–8Google Scholar
  44. 44.
    Vogiatzis G, Hernández C (2012) Self-calibrated, multi-spectral photometric stereo for 3d face capture. Int J Comput Vision 97(1):91–103CrossRefGoogle Scholar
  45. 45.
    Wagadarikar AA, Pitsianis NP, Sun X, Brady DJ (2009) Video rate spectral imaging using a coded aperture snapshot spectral imager. Opt Express 17(8):6368–6388CrossRefGoogle Scholar
  46. 46.
    Wu TP, Tang KL, Tang CK, Wong TT (2006) Dense photometric stereo: a markov random field approach. IEEE Trans Pattern Anal Mach Intell 28(11):1830–1846CrossRefGoogle Scholar
  47. 47.
    Zilly F, Riechert C, Mller M, Eisert P, Sikora T, Kauff P (2013) Real-time generation of multi-view video plus depth content using mixed narrow and wide baseline. J Visual Commun Image Represent 25(4):632–648CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2016

Authors and Affiliations

  1. 1.Computer Science DepartmentKAISTYuseong-guKorea

Personalised recommendations