Skip to main content
SpringerLink
Log in

A Switching Morphological Algorithm for Depth Map Recovery

  • Conference paper
  • First Online:
Analysis of Images, Social Networks and Texts (AIST 2019)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11832))

  • 952 Accesses

Abstract

In this paper, we propose a switching morphological filter for RGB-D depth map recovery. The switching algorithm consists of the following steps: detection of noisy pixels and hollow areas (holes) using morphological filtering; correction of the detected noisy and hole pixels. With the help of computer simulation, we show that the proposed algorithm is able to fast recover depth maps. So, the accuracy of 3D surface reconstruction with the proposed filtering noticeably increases. The performance of the proposed algorithm is compared in terms of the accuracy of 3D surface reconstruction and processing time with that of common depth filtering algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Zheng, L., Li, G., Sha, J.: The survey of medical image 3D reconstruction. In: Proceedings of SPIE, vol. 6534, pp. 6534–6536 (2007)

    Google Scholar 

  2. Echeagaray-Patron, B.A., Kober, V.: 3D face recognition based on matching of facial surfaces. In: Proceedings of SPIE, vol. 9598, p. 95980V-8 (2015)

    Google Scholar 

  3. Lee, K., Nguyen, T.Q.: Realistic surface geometry reconstruction using a hand-held RGB-D camera. Mach. Vis. Appl. 27(3), 377–385 (2016)

    Article  Google Scholar 

  4. Echeagaray-Patron, B.A., Kober, V.: Face recognition based on matching of local features on 3D dynamic range sequences. In: Proceedings of SPIE, vol. 9971, pp. 9971–9976 (2016)

    Google Scholar 

  5. Echeagaray-Patrón, B.A., Kober, V.I., Karnaukhov, V.N., Kuznetsov, V.V.: A method of face recognition using 3D facial surfaces. J. Commun. Technol. Electron. 62(6), 648–652 (2017)

    Article  Google Scholar 

  6. Ruchay, A., Dorofeev, K., Kober, A.: 3D object reconstruction using multiple Kinect sensors and initial estimation of sensor parameters. In: Proceedings of SPIE, vol. 10752, pp. 1075222–1075228 (2018)

    Google Scholar 

  7. Ruchay, A., Dorofeev, K., Kolpakov, V.: Fusion of information from multiple Kinect sensors for 3D object reconstruction. Comput. Opt. 42(5), 898–903 (2018)

    Article  Google Scholar 

  8. Tihonkih, D., Makovetskii, A., Voronin, A.: A modified iterative closest point algorithm for noisy data. In: Proceedings of SPIE, vol. 10396, pp. 10396–10397 (2017)

    Google Scholar 

  9. Makovetskii, A., Voronin, S., Kober, V.: An efficient algorithm of 3D total variation regularization. In: Proceedings of SPIE - The International Society for Optical Engineering, vol. 10752, p. 107522V (2018)

    Google Scholar 

  10. Voronin, S., Makovetskii, A., Voronin, A., Diaz-Escobar, J.: A regularization algorithm for registration of deformable surfaces. In: Proceedings of SPIE - The International Society for Optical Engineering, vol. 10752, p. 107522S (2018)

    Google Scholar 

  11. Boubou, S., Narikiyo, T., Kawanishi, M.: Adaptive filter for denoising 3D data captured by depth sensors. In: 2017 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON), pp. 1–4 (2017)

    Google Scholar 

  12. Zhang, X., Wu, R.: Fast depth image denoising and enhancement using a deep convolutional network. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2499–2503 (2016)

    Google Scholar 

  13. Milani, S., Calvagno, G.: Correction and interpolation of depth maps from structured light infrared sensors. Sig. Process. Image Commun. 41, 28–39 (2016)

    Article  Google Scholar 

  14. Fu, J., Wang, S., Lu, Y., Li, S., Zeng, W.: Kinect-like depth denoising. In: 2012 IEEE International Symposium on Circuits and Systems, pp. 512–515 (2012)

    Google Scholar 

  15. Lin, B.S., Chou, W.R., Yu, C., Cheng, P.H., Tseng, P.J., Chen, S.J.: An effective spatial-temporal denoising approach for depth images. In: 2015 IEEE International Conference on Digital Signal Processing (DSP), pp. 647–651 (2015)

    Google Scholar 

  16. Makovetskii, A., Voronin, S., Kober, V.: An efficient algorithm for total variation denoising. In: Ignatov, D.I., et al. (eds.) AIST 2016. CCIS, vol. 661, pp. 326–337. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-52920-2_30

    Chapter  Google Scholar 

  17. Moser, B., Bauer, F., Elbau, P., Heise, B., Schoner, H.: Denoising techniques for raw 3D data of ToF cameras based on clustering and wavelets. In: Proceedings of SPIE, vol. 6805, pp. 6805–6812 (2008)

    Google Scholar 

  18. Frank, M., Plaue, M., Hamprecht, F.A.: Denoising of continuous-wave time-of-flight depth images using confidence measures. Opt. Eng. 48(7), 077003 (2009)

    Article  Google Scholar 

  19. Kopf, J., Cohen, M.F., Lischinski, D., Uyttendaele, M.: Joint bilateral upsampling. ACM Trans. Graph. 26, 3 (2007)

    Google Scholar 

  20. Georgiev, M., Gotchev, A., Hannuksela, M.: Real-time denoising of ToF measurements by spatio-temporal non-local mean filtering. In: 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW), pp. 1–6 (2013)

    Google Scholar 

  21. Bhattacharya, S., Venkatesh, K.S., Gupta, S.: Depth filtering using total variation based video decomposition. In: 2015 Third International Conference on Image Information Processing (ICIIP), pp. 23–26 (2015)

    Google Scholar 

  22. Lei, J., Zhang, C., Wu, M., You, L., Fan, K., Hou, C.: A divide-and-conquer hole-filling method for handling disocclusion in single-view rendering. Multimed. Tools Appl. 76(6), 7661–7676 (2017)

    Article  Google Scholar 

  23. Zhang, Q., Chen, M., Zhu, H., Wang, X., Gan, Y.: An efficient depth map filtering based on spatial and texture features for 3D video coding. Neurocomputing 188, 82–89 (2016)

    Article  Google Scholar 

  24. Chen, R., Liu, X., Zhai, D., Zhao, D.: Depth image denoising via collaborative graph Fourier transform. In: Zhai, G., Zhou, J., Yang, X. (eds.) IFTC 2017. CCIS, vol. 815, pp. 128–137. Springer, Singapore (2018). https://doi.org/10.1007/978-981-10-8108-8_12

    Chapter  Google Scholar 

  25. Fu, M., Zhou, W.: Depth map super-resolution via extended weighted mode filtering. In: 2016 Visual Communications and Image Processing (VCIP), pp. 1–4 (2016)

    Google Scholar 

  26. Pourazad, M.T., Zhou, D., Lee, K., Karimifard, S., Ganelin, I., Nasiopoulos, P.: Improving depth map compression using a 3-phase depth map correction approach. In: 2015 IEEE International Conference on Multimedia Expo Workshops (ICMEW), pp. 1–6 (2015)

    Google Scholar 

  27. Liu, S., Chen, C., Kehtarnavaz, N.: A computationally efficient denoising and hole-filling method for depth image enhancement. In: Proceedings of SPIE, vol. 9897, pp. 9897–9899 (2016)

    Google Scholar 

  28. Paris, S., Kornprobst, P., Tumblin, J.: Bilateral Filtering. Now Publishers Inc., Hanover (2009)

    MATH  Google Scholar 

  29. Durand, F., Dorsey, J.: Fast bilateral filtering for the display of high-dynamic-range images. ACM Trans. Graph. 21(3), 257–266 (2002)

    Article  Google Scholar 

  30. Petschnigg, G., Agrawala, M., Hoppe, H., Szeliski, R., Cohen, M., Toyama, K.: Digital photography with flash and no-flash image pairs. ACM Trans. Graph. 23(3), 664–672 (2004)

    Article  Google Scholar 

  31. Jakhar, A., Sharma, S.: A novel approach for image enhancement using morphological operators. Int. J. Adv. Res. Comput. Sci. Technol. (IJARCST) 2, 300–302 (2014)

    Google Scholar 

  32. Yoshitaka, K.: Mathematical morphology-based approach to the enhancement of morphological features in medical images. J. Clin. Bioinform. 1, 33 (2011)

    Article  Google Scholar 

  33. Alexiou, E., Ebrahimi, T.: On subjective and objective quality evaluation of point cloud geometry. In: 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX), pp. 1–3 (2017)

    Google Scholar 

  34. Chan, D., Buisman, H., Theobalt, C., Thrun, S.: A noise-aware filter for real-time depth upsampling. In: Workshop on Multi-camera and Multi-modal Sensor Fusion Algorithms and Applications (2008)

    Google Scholar 

  35. Min, D., Lu, J., Do, M.N.: Depth video enhancement based on weighted mode filtering. IEEE Trans. Image Process. 21(3), 1176–1190 (2012)

    Article  MathSciNet  Google Scholar 

  36. Liu, J., Gong, X.: Guided depth enhancement via anisotropic diffusion. In: Huet, B., Ngo, C.-W., Tang, J., Zhou, Z.-H., Hauptmann, A.G., Yan, S. (eds.) PCM 2013. LNCS, vol. 8294, pp. 408–417. Springer, Cham (2013). https://doi.org/10.1007/978-3-319-03731-8_38

    Chapter  Google Scholar 

  37. Diebel, J., Thrun, S.: An application of Markov random fields to range sensing. In: Proceedings of the 18th International Conference on Neural Information Processing Systems, NIPS 2005, pp. 291–298 (2005)

    Google Scholar 

  38. Harrison, A., Newman, P.: Image and sparse laser fusion for dense scene reconstruction. In: Howard, A., Iagnemma, K., Kelly, A. (eds.) Field and Service Robotics. Springer Tracts in Advanced Robotics, vol. 62, pp. 219–228. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-13408-1_20

    Chapter  Google Scholar 

  39. Yang, Q., Yang, R., Davis, J., Nister, D.: Spatial-depth super resolution for range images. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)

    Google Scholar 

  40. Newcombe, R.A., et al.: KinectFusion: real-time dense surface mapping and tracking. In: IEEE ISMAR (2011)

    Google Scholar 

  41. Fuhrmann, S., Goesele, M.: Fusion of depth maps with multiple scales. ACM Trans. Graph. 30(6), 148:1–148:8 (2011)

    Article  Google Scholar 

  42. Okada, M., Ishikawa, T., Ikegaya, Y.: A computationally efficient filter for reducing shot noise in low S/N data. PLoS ONE 11(6), e0157595 (2016)

    Article  Google Scholar 

  43. Yuan, C., Li, Y.: Switching median and morphological filter for impulse noise removal from digital images. Optik Int. J. Light Electron Opt. 126(18), 1598–1601 (2015)

    Article  Google Scholar 

  44. Ruchay, A., Kober, V.: Impulsive noise removal from color images with morphological filtering. In: van der Aalst, W.M.P., et al. (eds.) AIST 2017. LNCS, vol. 10716, pp. 280–291. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-73013-4_26

    Chapter  Google Scholar 

  45. Soille, P.: Morphological Image Analysis: Principles and Applications, 2nd edn. Springer, Heidelberg (2004). https://doi.org/10.1007/978-3-662-05088-0

    Book  MATH  Google Scholar 

  46. Najman, L., Talbot, H.: Mathematical Morphology: From Theory to Applications. ISTE-Wiley, Hoboken (2010)

    MATH  Google Scholar 

  47. Rusu, R.B., Cousins, S.: 3D is here: point cloud library (PCL). In: 2011 IEEE International Conference on Robotics and Automation, pp. 1–4 (2011)

    Google Scholar 

Download references

Acknowledgments

This work was supported by the Russian Science Foundation, grant no. 17-76-20045.

Author information

Authors and Affiliations

  1. Federal Research Centre of Biological Systems and Agro-Technologies of the Russian Academy of Sciences, Orenburg, Russian Federation

    Alexey N. Ruchay

  2. Department of Mathematics, Chelyabinsk State University, Chelyabinsk, Russian Federation

    Alexey N. Ruchay, Konstantin A. Dorofeev & Vsevolod V. Kalschikov

Authors
  1. Alexey N. Ruchay
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Konstantin A. Dorofeev
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vsevolod V. Kalschikov
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alexey N. Ruchay .

Editor information

Editors and Affiliations

  1. RWTH Aachen University, Aachen, Germany

    Wil M. P. van der Aalst

  2. University of Ljubljana, Ljubljana, Slovenia

    Vladimir Batagelj

  3. National Research University Higher School of Economics, Moscow, Russia

    Dmitry I. Ignatov

  4. Krasovskii Institute of Mathematics and Mechanics, Yekaterinburg, Russia

    Michael Khachay

  5. National Research University Higher School of Economics, Moscow, Russia

    Valentina Kuskova

  6. University of Oslo, Oslo, Norway

    Andrey Kutuzov

  7. National Research University Higher School of Economics, Moscow, Russia

    Sergei O. Kuznetsov

  8. National Research University Higher School of Economics, Moscow, Russia

    Irina A. Lomazova

  9. Lomonosov Moscow State University, Moscow, Russia

    Natalia Loukachevitch

  10. LORIA, Vandœuvre-lès-Nancy, France

    Amedeo Napoli

  11. University of Florida, Gainesville, FL, USA

    Panos M. Pardalos

  12. Ca Foscari University of Venice, Venice, Italy

    Marcello Pelillo

  13. National Research University Higher School of Economics, Nizhny Novgorod, Russia

    Andrey V. Savchenko

  14. Kazan Federal University, Kazan, Russia

    Elena Tutubalina

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ruchay, A.N., Dorofeev, K.A., Kalschikov, V.V. (2019). A Switching Morphological Algorithm for Depth Map Recovery. In: van der Aalst, W., et al. Analysis of Images, Social Networks and Texts. AIST 2019. Lecture Notes in Computer Science(), vol 11832. Springer, Cham. https://doi.org/10.1007/978-3-030-37334-4_32

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-37334-4_32

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-37333-7

  • Online ISBN: 978-3-030-37334-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation