Building Models from Sensor Data: An Application Shared by the Computer Vision and the Computer Graphics Community

  • Gerhard Roth
Part of the NATO Science Series book series (ASHT, volume 84)


The problem of building virtual models from sensor data increases in importance as powerful graphics rendering hardware becomes widespread. Model building stands at the interface between computer vision and computer graphics, and researchers from both areas have made contributions. We believe that only by a systematic review of the remaining open research question can further progress be made. This paper is an attempt at providing such a review. First, we describe the basic steps in the model building pipeline. Then we discuss the open problems that remain in each step. Finally, we describe some overall research themes that we believe should guide further work in this area.


Computer Graphic Sensor Data Active Sensor Iterative Close Point Correspondence Problem 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    A. P. Ashbrook, R. B. Fisher, C. Robertson, and N. Werghi. Finding surface correspondence for object recognition using pairwise geometric histograms. In Computer Vision-ECCV’98, pages 674–686, Freiburg, Germany, June 1998.CrossRefGoogle Scholar
  2. [2]
    D. Ballard and C. Brown. Computer vision. Prentice Hall, 1982.Google Scholar
  3. [3]
    R. Baribeau, M. Rioux, and G. Godin. Color reflectance modeling using a polychromatic laser sensor. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2):263–269, 1992.CrossRefGoogle Scholar
  4. [4]
    R. Benjemaa and F. Schmitt. A solution for the registration of multiple 3D point sets using unit quaternions. In Computer Vision-ECCV’98, pages 34–50, 1998.CrossRefGoogle Scholar
  5. [5]
    J. Beraldin, F. Biais, J. Cournoyer, M. Rioux, F. Biais, S.F. El-Hakim, and G. Godin. Object model creation from multiple range images: acquisition, calibration, model building and verification. In International Conference on Recent Advances in 3-D Digital Imaging and Modelling, pages 326–333, Ottawa, Canada, 1997.Google Scholar
  6. [6]
    R. Bergevin, D. Laurendeau, and D. Poussart. Registering range views of multipart objects. Computer Vision and Image Understanding, 61(1):1–16, January 1995.CrossRefGoogle Scholar
  7. [7]
    F. Bernardini, C. Bajaj, J. Chen, and D. Schikore. Automatic reconstruction of CAD models from digital scans. International Journal of Computational Geometry and Applications, 9(4):327–330, August 1999.CrossRefGoogle Scholar
  8. [8]
    P. J. Besl. Active, optical range imaging sensors. Machine Vision and Applications, 1(1):127–152, 1988.CrossRefGoogle Scholar
  9. [9]
    P. J. Besl and N. D. McKay. A method for registration of 3-D shapes. IEEE Trans, on Pattern Analysis and Machine Intelligence, 14(2):239–256, Feb. 1992.CrossRefGoogle Scholar
  10. [10]
    E. Chen and L. Williams. View interpolation for image synthesis. In Computer Graphics: Siggraph, pages 279–288, 1993.Google Scholar
  11. [11]
    Y. Chen and G. Medioni. Description of complex objects from multiple range images using an inflating balloon model. Computer Vision and Image Understanding, 61(3):325–334, May 1995.CrossRefGoogle Scholar
  12. [12]
    C-S. Cheng, Y-P. Hung, and J-B. Chung. A fast automatic method for registration of partially overlapping range images. In International Conference on Computer Vision, pages 242–248, Bombay, India, 1998.Google Scholar
  13. [13]
    A. Ciampalini, P. Cignoni, C. Montani, and R. Scopigno. Multiresolut ion decimation based on global error. Visual Computer, 13:228–246, 1997.CrossRefGoogle Scholar
  14. [14]
    B. Cur less and M. Levoy. A volumetric method for building complex models from range images. In Computer Graphics: Siggraph’96 Proceedings, pages 221–227, 1996.Google Scholar
  15. [15]
    S. F. El-Hakim, C. Brenner, and G. Roth. A multi-sensor approach to creating accurate virtual environments. ISPRS Journal of Photogrammetry and Remote Sensing, 53(6):379–391, December 1998.CrossRefGoogle Scholar
  16. [16]
    O. Faugeras. Three-dimensional computer vision. The MIT Press, 1996.Google Scholar
  17. [17]
    R. Fisher, A. Fitzgibbon, and D. Eggert. Extracting surface patches from complete range descriptions. In International Conference on Recent Advances in 3-D Digital Imaging and Modelling, pages 148–157, Ottawa, Canada, May 1997. IEEE Press.Google Scholar
  18. [18]
    J. D. Foley and A. Van Dam. Fundamentals of Interactive Computer Graphics. Addison-Wesley, Reading, Mass., 1982.Google Scholar
  19. [19]
    E. Gagnon, J.-F. Rivest, M. Greenspan, and N. Burtnyk. A computer assisted range image registration system for nuclear waste cleanup. In IEEE Instrumentation and Measurement Conference, pages 224–229, Brussels, Belgium, June 1996.Google Scholar
  20. [20]
    W. Grimson and J. Mundy. Computer vision applications. Communications of the ACM, 37(3):45–51, Mar. 1994.CrossRefGoogle Scholar
  21. [21]
    R. I. Hartley. Self-calibration of stationary cameras. International Journal of Computer Vision, 22(1):5–23, February 1997.CrossRefGoogle Scholar
  22. [22]
    M. Hirose. Image-based virtual world generation. IEEE Multi-Media, pages 27–33, Jan. 1997.Google Scholar
  23. [23]
    H. Hoppe. Progressive meshes. In Computer Graphics: Siggraph’96 Proceedings, pages 225–235, 1996.Google Scholar
  24. [24]
    H. Hoppe, T. DeRose, T. Duchamp, J. McDonald, and W. Stuetzle. Surface reconstruction from unorganized data points. In Computer Graphics 26: Siggraph’92 Conference Proceedings, volume 26, pages 71–78, July 1992.Google Scholar
  25. [25]
    P. Jasiobedski. Fusing and guiding range measurements with colour video images. In Proceedings International Conference on Recent Advances in 3-D Digital Imaging and Modelling, pages 339–347, Ottawa, Ontario, 1997. IEEE Computer Society Press.Google Scholar
  26. [26]
    S. B. Kang. A survey of image-based rendering techniques. Technical Report CRL 97/4, Digital Equipment Corporation, Cambridge Research Lab., Cambridge, MA, 1997.Google Scholar
  27. [27]
    R. Klette, K. Schluns, and A. Koschan. Computer Vision: three-dimensional data from images. Springer, 1996.Google Scholar
  28. [28]
    V. Koivunen and R. Bajcsy. Geometric methods for building CAD models from range data. In Geometric Methods in Computer Vision II, volume 2031, pages 205–216. SPIE, The International Society for Optical Engineering, July 1993.Google Scholar
  29. [29]
    M. Levoy. The digital Michelangelo project. Technical Report (see, Stanford University, Computer Graphics Laboratory, 1998.Google Scholar
  30. [30]
    A. D. Marshall and R. R. Martin. Computer vision, models and inspection. World Scientific, 1992.Google Scholar
  31. [31]
    J. Maver and R. Bajcsy. Occlusion as a guide for planning the next best view. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(5):417–433, May 1993.CrossRefGoogle Scholar
  32. [32]
    L. McMillan. An image-based approach to three-dimensional computer graphics. PhD thesis, Univ. of North Carolina at Chapel Hill, 1997.Google Scholar
  33. [33]
    K. Ng, V. Sequeira, S. Butterfield, D. Hogg, and J. G. M. Goncalves. An integrated multi-sensory system for photo-realistic 3D scene reconstruction. In ISPRS Commission V Symposium: Real-Time Imaging and Dynamic Analysis, volume XXXII, pages 356–363, 1998.Google Scholar
  34. [34]
    L. Nyland and D. McAllister. The impact of dense range data on computer graphics. In Proceedings of multi-view analysis and modelling workshop, Fort Collins, June 1999.Google Scholar
  35. [35]
    M. Petrov, A. Talapov, and T. Robertson. Optical 3D digitizers: bringing life to the virtual world. IEEE Computer Graphics and Applications, pages 28–37, May/June 1998.Google Scholar
  36. [36]
    R. Pito and R. Bajcsy. A solution to the next best view problem for automated CAD model acquisition. In SPIE, volume 2596, pages 78–89, 1995.Google Scholar
  37. [37]
    R. Pito and R. Bajcsy. Data acquisition and representation of mechanical parts and interfaces to manufacturing devices. In International Conference on Recent Advances in 3-D Digital Imaging and Modelling, pages 2–9, Ottawa, Ontario, Canada, 1997.Google Scholar
  38. [38]
    M. Pollefeys, R. Koch, M. Vergauwen, and L. VanGool. Automatic generation of 3D models from photographs. In Proceedings Virtual Systems and MultiMedia, 1998.Google Scholar
  39. [39]
    M. Reed, P. Allen, and S. Stamos. 3-D modelling from range imagery: an incremental method with a planning component. In International Conference on Recent Advances in 3-D Digital Imaging and Modelling, pages 76–83. IEEE Press, 1997.Google Scholar
  40. [40]
    G. Roth. An automatic registration algorithm for two overlapping range images. In F. Solina and A. Leonardis, editors, Computer Analysis of Images and Patterns, 8th International Conference CAIP’99, Ljubljana, Slovenia, September 1999, volume 1689 of Lecture Notes in Computer Science, pages 329–338. Springer, 1999.Google Scholar
  41. [41]
    G. Roth and E. Wibowo. An efficient volumetric method for building closed triangular meshes from 3-D image and point data. In Graphics Interface 97, pages 173–180, Kelowna, BC, Canada, May 1997.Google Scholar
  42. [42]
    Y. Sato and K. Ikeuchi. Reflectance analysis for 3d computer graphics model generation. Graphical models and image processing, 58(5):437–451, September 1996.CrossRefGoogle Scholar
  43. [43]
    H.-Y. Shum and R. Szeliski. Panoramic image mosaics. Technical Report MSR-TR-97–23, Microsoft Research, Redmond, WA, 1997.Google Scholar
  44. [44]
    R. Szeliski. Video mosaics for virtual environments. IEEE Computer Graphics and Applications, pages 22–30, March 1996.Google Scholar
  45. [45]
    P. Torr. Motion segmentation and outlier detection. PhD thesis, University of Oxford, 1995.Google Scholar
  46. [46]
    P. Torr and D. Murray. Outlier detection and motion segmentation. In Sensor Fusion VI, volume 2059, pages 432–443, 1993.Google Scholar
  47. [47]
    G. Turk and M. Levoy. Zippered polygon meshes from range images. In Computer Graphics (Siggraph’94), volume 26, pages 311–318, 1994.Google Scholar
  48. [48]
    T. Werner, T. Pajdla, and V. Hlavač. Efficient 3-D scene visualization by image extrapolation. In Proceedings European Conference on Computer Vision, pages 382–296, 1998.Google Scholar
  49. [49]
    J. C. Xia, J. El-Sana, and A. Varshney. Adaptive real-time level-of-detail-based rendering for polygonal models. IEEE Transactions on Visualization and Graphics, 3(2):171–183, Apr. 1997.CrossRefGoogle Scholar
  50. [50]
    Z. Zhang. Determining the epipolar geometry and its uncertainty: a review. International Journal of Computer Vision, 27(2), March 1998.Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2000

Authors and Affiliations

  • Gerhard Roth

There are no affiliations available

Personalised recommendations