Neighborhood Filters and the Recovery of 3D Information

  • Julie Digne
  • Mariella Dimiccoli
  • Philippe Salembier
  • Neus Sabater
Reference work entry


Following their success in image processing (see  Chap. 26), neighborhood filters have been extended to 3D surface processing. This adaptation is not straightforward. It has led to several variants for surfaces depending on whether the surface is defined as a mesh, or as a raw data point set. The image gray level in the bilateral similarity measure is replaced by a geometric information such as the normal or the curvature. The first section of this chapter reviews the variants of 3D mesh bilateral filters and compares them to the simplest possible isotropic filter, the mean curvature motion.

In a second part, this chapter reviews applications of the bilateral filter to a data composed of a sparse depth map (or of depth cues) and of the image on which they have been computed. Such sparse depth cues can be obtained by stereo vision or by psychophysical techniques. The underlying assumption to these applications is that pixels with similar intensity around a region are likely to have similar depths. Therefore, when diffusing depth information with a bilateral filter based on locality and color similarity, the discontinuities in depth are assured to be consistent with the color discontinuities, which is generally a desirable property. In the reviewed applications, this ends up with the reconstruction of a dense perceptual depth map from the joint data of an image and of depth cues.


Point Cloud Bilateral Filter Curvature Motion Regression Plane Stereo Match Algorithm 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The David raw point set is courtesy of the Digital Michelangelo Project, Stanford University. Fragment “31u” of the Stanford Fragment Urbis Romae, Stanford University, the Screw Nut point set, is provided by the AIM@SHAPE repository and is courtesy of Laurent Saboret, INRIA. The research is partially financed by Institut Farman, ENS Cachan, the Centre National d’Etudes Spatiales (MISS Project), the European Research Council (advanced grant Twelve Labours), and the Office of Naval research (grant N00014-97-1-0839).

References and Further Reading

  1. 1.
    Adams A, Gelfand N, Dolson J, Levoy M (2009) Gaussian kd-trees for fast high-dimensional filtering. ACM Trans Graph 28(3):1–12CrossRefGoogle Scholar
  2. 2.
    Ansar A, Castano A, Matthies L (2004) Enhanced real-time stereo using bilateral filtering. In: 3DPVT ’04: Proceedings of the 3D data processing, visualization, and transmission, 2nd international symposium. IEEE Computer Society, Washington, pp 455–462Google Scholar
  3. 3.
    Buades A, Coll B, Morel JM (2005) A review of image denoising algorithms, with a new one. Multiscale Model Simul 4(2):490–530MathSciNetMATHCrossRefGoogle Scholar
  4. 4.
    Buades T, Coll B, Morel J-M (2006) Neighborhood filters and pde’s. Numerische Mathematik 105(1):11–34MathSciNetCrossRefGoogle Scholar
  5. 5.
    Choudhury P, Tumblin J (2005) The trilateral filter for high contrast images and meshes. In: SIGGRAPH ’05: ACM SIGGRAPH 2005 Courses. ACM, New York, p 5CrossRefGoogle Scholar
  6. 6.
    Delage E, Lee H, Ng Y (2006) A dynamic bayesian network model for autonomous 3d reconstruction from a single indoor image. In: International conference on computer vision and pattern recognition (CVPR), pp 1–8Google Scholar
  7. 7.
    Delon J, Rougé B (2007) Small baseline stereovision. J Math Imaging Vis 28(3):209–223CrossRefGoogle Scholar
  8. 8.
    Digne J, Morel J-M, Mehdi-Souzani C, Lartigue C (October 2009) Scale space meshing of raw data point sets. Preprint CMLA 2009-30 - ENS CachanGoogle Scholar
  9. 9.
    Dimiccoli M (2009) Monocular depth estimation for image segmentation and filtering. Ph.D thesis, Technical University of Catalonia (UPC), CataloniaGoogle Scholar
  10. 10.
    Dimiccoli M, Morel JM, Salembier P (December 2008) Monocular depth by nonlinear diffusion. In: Indian conference on computer vision, graphics and image processing (ICVGIP), BhubaneswarGoogle Scholar
  11. 11.
    Dunn JC (1973) A fuzzy relative of the isodata process and its use in detecting compact well-separated clusters. J Cybernetics 3(3):32–57MathSciNetMATHCrossRefGoogle Scholar
  12. 12.
    Facciolo G, Caselles V (2009) Geodesic neighborhoods for piecewise affine interpolation of sparse data. In: International conference on image processingGoogle Scholar
  13. 13.
    Facciolo G, Lecumberry F, Almansa A, Pardo A, Caselles V, Rougé B (2006) Constrained anisotropic diffusion and some applications. In: British machine vision conferenceGoogle Scholar
  14. 14.
    Fleishman S, Drori I, Cohen-Or D (2003) Bilateral mesh denoising. ACM Trans Graph 22(3): 950–953CrossRefGoogle Scholar
  15. 15.
    Gamble E, Poggio T (1987) Visual integration and detection of discontinuities: the key role of intensitiy edges. Technical Report 970, MIT AI Lab MemoGoogle Scholar
  16. 16.
    Gehrig SK, Franke U (2007) Improving stereo sub-pixel accuracy for long range stereo. In: Proceedings of the 11th international journal of computer vision, pp 1–7Google Scholar
  17. 17.
    Hoiem D, Stein AN, Efros AA, Hebert M (2007) Recovering occlusion boundaries from a single image. In: Proceedings of international conference on computer vision (ICCV), pp 1–8Google Scholar
  18. 18.
    Hou Q, Bai L, Wang Y (2005) Mesh smoothing via adaptive bilateral filtering. In: Springer (ed) Computational science - ICCS 2005. Springer, Berlin, pp 273–280Google Scholar
  19. 19.
    Jones TR, Durand F, Zwicker M (2004) Normal improvement for point rendering. IEEE Comput Graph Appl 24(4):53–56CrossRefGoogle Scholar
  20. 20.
    Jones TR, Durand F, Desbrun M (2003) Non-iterative, feature preserving mesh smoothing. In: SIGGRAPH ’03: ACM SIGGRAPH 2003 papers. ACM, New York, pp 943–949Google Scholar
  21. 21.
    Kellman PJ, Shipley TF (1991) Visual interpolation in object perception. Curr Directions in Psychol Sci 1(6):193–199CrossRefGoogle Scholar
  22. 22.
    Koller D, Trimble J, Najbjerg T, Gelfand N, Levoy M (2006) Fragments of the city: tanford’s digital forma urbis romae project. In: Proceedings of 3rd Williams symposium on classical architecture. J Roman Archaeol (Suppl 61), pp 237–252Google Scholar
  23. 23.
    Lafarge F, Descombes X, Zerubia J, Pierrot-Deseilligny M (2008) Automatic building extraction from dems using an object approach and application to the 3d-city modeling. J Photogramm Remote Sens 63(3):365–381CrossRefGoogle Scholar
  24. 24.
    Lee J-S (1983) Digital image smoothing and the sigma filter. Comput Vision Graph Imag Process 24(2):255–269CrossRefGoogle Scholar
  25. 25.
    Liu Y-S, Yu P-Q, Yong J-H, Zhang H, Sun J-G (2005) Bilateral filter for meshes using new predictor. In: Springer (ed) Lecture notes in computer science, computational and information science, vol 3314/2005. Springer, Heidelberg, pp 1093–1099Google Scholar
  26. 26.
    Mattoccia S, Giardino S, Gambin A (2009) Accurate and efficient cost aggregation strategy for stereo correspondence based on approximated joint bilateral filtering. In: Asian conference on computer vision (ACCV09)Google Scholar
  27. 27.
    Metzger W (1975) Gesetze des Sehens. Waldemar Kramer, FrankfurtGoogle Scholar
  28. 28.
    Miropolsky A, Fischer A (2004) Reconstruction with 3d geometric bilateral filter. In: SM ’04: Proceedings of the 9th ACM symposium on solid modeling and applications. Eurographics Association, Aire-la-Ville, Switzerland, pp 225–229Google Scholar
  29. 29.
    Ohtake Y, Belyaev AG, Seidel H-P (2002) Mesh smoothing by adaptive and anisotropic gaussian filter applied to mesh normals. In: VMV, pp 203–210Google Scholar
  30. 30.
    Paris S, Durand F (2009) A fast approximation of the bilateral filter using a signal processing approach. Int J Comput Vision 81(1):24–52CrossRefGoogle Scholar
  31. 31.
    Paris S, Kornprobst P, Tumblin J, Durand F (2009) Bilateral filtering: theory and applications, vol 4. Foundations and Trends® in Computer Graphics and Vision. IEEE Transactions on Visualization and Computer GraphicsGoogle Scholar
  32. 32.
    Rother D, Sapiro G (2009) 3d reconstruction from a single image (submitted to IEEE transactions on pattern analysis and machine intelligence IMA Prepr InternationalGoogle Scholar
  33. 33.
    Sabater N (2009) Reliability and accuracy in stereovision. Application to aerial and satellite high resolution image. Ph.D. thesis, ENS CachanGoogle Scholar
  34. 34.
    Scharstein D, Szeliski R (2002) A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int J Computer Vis 47(1–3): 7–42MATHCrossRefGoogle Scholar
  35. 35.
    Tomasi C, Manduchi R (1998) Bilateral filtering for gray and color images. In: ICCV ’98: Proceedings of the 6th international conference on computer vision, IEEE Computer Society, Washington, p 839Google Scholar
  36. 36.
    Wang C (2006) Bilateral recovering of sharp edges on feature-insensitive sampled meshes. IEEE Trans Visual Comput Graph 12(4): 629–639CrossRefGoogle Scholar
  37. 37.
    Wang L, Liao M, Gong M, Yang R, Nistér D (2006) High-quality real-time stereo using adaptive cost aggregation and dynamic programming. In: 3rd international symposium on 3d data processing, visualization and transmission, 3DPVTGoogle Scholar
  38. 38.
    Wang L, Yuan B, Chen J (2006) Robust fuzzy c-means and bilateral point clouds denoising. In: 2006 8th international conference on signal processing, vol 2, pp 16–20Google Scholar
  39. 39.
    Wang R-F, Zhang S-Y, Zhang Y, Ye X-Z (June 2008) Similarity-based denoising of point-sampled surfaces. J Zhejiang Univ 9(6):807–815MATHCrossRefGoogle Scholar
  40. 40.
    Yang Q, Wang L, Yang R, Stewénius H, Nistér D (2006) Stereo matching with color-weighted correlation, hierarchical belief propagation and occlusion handling. IEEE Trans Pattern Anal Mach Intell (PAMI) 31(3):1–13Google Scholar
  41. 41.
    Yang Q, Yang R, Davis J, Nistér D (2007) Spatial-depth super resolution for range images. In: International conference on computer vision and pattern recognition (CVPR)Google Scholar
  42. 42.
    Yaroslavsky LP (1985) Digital picture processing. An introduction, vol 9 of Springer series in information sciences. Springer, BerlinGoogle Scholar
  43. 43.
    Yin J, Cooperstock JR (2004) Improving depth maps by nonlinear diffusion. In: Proceedings of 12th international conference in Central Europe on computer graphics, visualization and computer vision (WSCG), pp 1–8Google Scholar
  44. 44.
    Yoon K-J, Kweon S (2006) Adaptive support-weight approach for correspondence search. IEEE Trans Pattern Anal Mach Intell 28(4): 650–656CrossRefGoogle Scholar
  45. 45.
    Yoshizawa S, Belyaev A, Seidel H-P (2006) Smoothing by example: mesh denoising by averaging with similarity-based weights. In: SMI’06: Proceedings of the IEEE international conference on shape modeling and applications 2006. IEEE Computer Society, Washington, p 9CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2011

Authors and Affiliations

  • Julie Digne
    • 1
  • Mariella Dimiccoli
    • 2
  • Philippe Salembier
    • 3
  • Neus Sabater
    • 4
  1. 1.École Normale Supérieure de CachanCachanFrance
  2. 2.Collège de FranceParisFrance
  3. 3.Technical University of Catalonia (UPC)BarcelonaSpain
  4. 4.École Normale Supérieure de CachanCachanFrance

Personalised recommendations