SRA: Fast Removal of General Multipath for ToF Sensors

  • Daniel Freedman
  • Yoni Smolin
  • Eyal Krupka
  • Ido Leichter
  • Mirko Schmidt
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8689)


A major issue with Time of Flight sensors is the presence of multipath interference. We present Sparse Reflections Analysis (SRA), an algorithm for removing this interference which has two main advantages. First, it allows for very general forms of multipath, including interference with three or more paths, diffuse multipath resulting from Lambertian surfaces, and combinations thereof. SRA removes this general multipath with robust techniques based on L 1 optimization. Second, due to a novel dimension reduction, we are able to produce a very fast version of SRA, which is able to run at frame rate. Experimental results on both synthetic data with ground truth, as well as real images of challenging scenes, validate the approach.


Canonical Transformation Mean Absolute Error Orthogonal Match Pursuit General Multipath Fast Removal 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Supplementary material

978-3-319-10590-1_16_MOESM1_ESM.pdf (161 kb)
Electronic Supplementary Material (PDF 161 KB)


  1. 1.
    Bhandari, A., Kadambi, A., Whyte, R., Barsi, C., Feigin, M., Dorrington, A., Raskar, R.: Resolving multipath interference in time-of-flight imaging via modulation frequency diversity and sparse regularization. Optics Letters 39(6), 1705–1708 (2014)CrossRefGoogle Scholar
  2. 2.
    Candes, E.J., Romberg, J.K., Tao, T.: Stable signal recovery from incomplete and inaccurate measurements. Communications on Pure and Applied Mathematics 59(8), 1207–1223 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  3. 3.
    Candes, E.J., Tao, T.: Decoding by linear programming. IEEE Transactions on Information Theory 51(12), 4203–4215 (2005)CrossRefzbMATHMathSciNetGoogle Scholar
  4. 4.
    Donoho, D.L.: Compressed sensing. IEEE Transactions on Information Theory 52(4), 1289–1306 (2006)CrossRefzbMATHMathSciNetGoogle Scholar
  5. 5.
    Dorrington, A.A., Godbaz, J.P., Cree, M.J., Payne, A.D., Streeter, L.V.: Separating true range measurements from multi-path and scattering interference in commercial range cameras. In: IS&T/SPIE Electronic Imaging, pp. 786404. International Society for Optics and Photonics (2011)Google Scholar
  6. 6.
    Falie, D., Buzuloiu, V.: Further investigations on ToF cameras distance errors and their corrections. In: European Conference on Circuits and Systems for Communications (ECCSC), pp. 197–200 (2008)Google Scholar
  7. 7.
    Falie, D., Buzuloiu, V.: Distance errors correction for the time of flight (ToF) cameras. In: IEEE International Workshop on Imaging Systems and Techniques, IST 2008, pp. 123–126. IEEE (2008)Google Scholar
  8. 8.
    Fuchs, S.: Multipath interference compensation in time-of-flight camera images. In: International Conference on Pattern Recognition (ICPR), pp. 3583–3586 (2010)Google Scholar
  9. 9.
    Fuchs, S., Suppa, M., Hellwich, O.: Compensation for multipath in ToF camera measurements supported by photometric calibration and environment integration. In: Chen, M., Leibe, B., Neumann, B. (eds.) ICVS 2013. LNCS, vol. 7963, pp. 31–41. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  10. 10.
    Godbaz, J.P., Cree, M.J., Dorrington, A.A.: Closed-form inverses for the mixed pixel/multipath interference problem in amcw lidar. In: IS&T/SPIE Electronic Imaging, pp. 829618. International Society for Optics and Photonics (2012)Google Scholar
  11. 11.
    Han, J., Shao, L., Xu, D., Shotton, J.: Enhanced computer vision with Microsoft Kinect sensor: A review. IEEE Transactions on Cybernetics 43(5), 1318–1334 (2013)CrossRefGoogle Scholar
  12. 12.
    Heide, F., Hullin, M.B., Gregson, J., Heidrich, W.: Low-budget transient imaging using photonic mixer devices. ACM Transactions on Graphics (TOG) 32(4), 45 (2013)Google Scholar
  13. 13.
    Jiménez, D., Pizarro, D., Mazo, M., Palazuelos, S.: Modelling and correction of multipath interference in time of flight cameras. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 893–900 (2012)Google Scholar
  14. 14.
    Kadambi, A., Whyte, R., Bhandari, A., Streeter, L., Barsi, C., Dorrington, A., Raskar, R.: Coded time of flight cameras: sparse deconvolution to address multipath interference and recover time profiles. ACM Transactions on Graphics (TOG) 32(6), 167 (2013)CrossRefGoogle Scholar
  15. 15.
    Kirmani, A., Benedetti, A., Chou, P.A.: Spumic: Simultaneous phase unwrapping and multipath interference cancellation in time-of-flight cameras using spectral methods. In: IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6 (2013)Google Scholar
  16. 16.
    Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Ma, Y.: Robust face recognition via sparse representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 31(2), 210–227 (2009)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Daniel Freedman
    • 1
  • Yoni Smolin
    • 1
  • Eyal Krupka
    • 1
  • Ido Leichter
    • 1
  • Mirko Schmidt
    • 2
  1. 1.Microsoft ResearchHaifaIsrael
  2. 2.Microsoft CorporationMountain ViewUSA

Personalised recommendations