Guided Image Super-Resolution: A New Technique for Photogeometric Super-Resolution in Hybrid 3-D Range Imaging

  • Florin C. Ghesu
  • Thomas KöhlerEmail author
  • Sven Haase
  • Joachim Hornegger
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8753)


In this paper, we augment multi-frame super-resolution with the concept of guided filtering for simultaneous upsampling of 3-D range data and complementary photometric information in hybrid range imaging. Our guided super-resolution algorithm is formulated as joint maximum a-posteriori estimation to reconstruct high-resolution range and photometric data. In order to exploit local correlations between both modalities, guided filtering is employed for regularization of the proposed joint energy function. For fast and robust image reconstruction, we employ iteratively re-weighted least square minimization embedded into a cyclic coordinate descent scheme. The proposed method was evaluated on synthetic datasets and real range data acquired with Microsoft’s Kinect. Our experimental evaluation demonstrates that our approach outperforms state-of-the-art range super-resolution algorithms while it also provides super-resolved photometric data.



The authors gratefully acknowledge funding of the Erlangen Graduate School in Advanced Optical Technologies (SAOT) by the German National Science Foundation (DFG) in the framework of the excellence initiative and the support by the DFG under Grant No. HO 1791/7-1.


  1. 1.
    Babacan, S.D., Molina, R., Katsaggelos, A.K.: Variational Bayesian super resolution. IEEE Trans. Image Process. 20(4), 984–999 (2011)MathSciNetCrossRefGoogle Scholar
  2. 2.
    Bauer, S., Seitel, A., Hofmann, H., Blum, T., Wasza, J., Balda, M., Meinzer, H.-P., Navab, N., Hornegger, J., Maier-Hein, L.: Real-time range imaging in health care: a survey. In: Grzegorzek, M., Theobalt, C., Koch, R., Kolb, A. (eds.) Time-of-Flight and Depth Imaging. LNCS, vol. 8200, pp. 228–254. Springer, Heidelberg (2013)Google Scholar
  3. 3.
    Beder, C., Bartczak, B., Koch, R.: A comparison of PMD-cameras and stereo-vision for the task of surface reconstruction using patchlets. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007)Google Scholar
  4. 4.
    Bhavsar, A.V., Rajagopalan, A.N.: Range map superresolution-inpainting, and reconstruction from sparse data. Comput. Vis. Image Underst. 116(4), 572–591 (2012)CrossRefGoogle Scholar
  5. 5.
    Cui, Y., Schuon, S., Chan, D., Thrun, S., Theobalt, C.: 3D shape scanning with a time-of-flight camera. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1173–1180 (2010)Google Scholar
  6. 6.
    Elad, M., Feuer, A.: Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images. IEEE Trans. Image Process. 6(12), 1646–1658 (1997)CrossRefGoogle Scholar
  7. 7.
    Farsiu, S., Robinson, D., Elad, M., Milanfar, P.: Advances and challenges in super-resolution. Int. J. Imaging Syst. Technol. 14, 47–57 (2004)CrossRefGoogle Scholar
  8. 8.
    Farsiu, S., Robinson, D., Elad, M., Milanfar, P.: Fast and robust multiframe super resolution. IEEE Trans. Image Process. 13(10), 1327–1344 (2004)CrossRefGoogle Scholar
  9. 9.
    He, K., Sun, J., Tang, X.: Guided image filtering. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part I. LNCS, vol. 6311, pp. 1–14. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  10. 10.
    Köhler, T., Haase, S., Bauer, S., Wasza, J., Kilgus, T., Maier-Hein, L., Feubner, H., Hornegger, J.: ToF meets RGB: novel multi-sensor super-resolution for hybrid 3-D endoscopy. Med. Image Comput. Comput. Assist. Interv. 16, 139–146 (2013)Google Scholar
  11. 11.
    Köhler, T., Haase, S., Bauer, S., Wasza, J., Kilgus, T., Maier-Hein, L., Feuner, H., Hornegger, J.: Outlier detection for multi-sensor super-resolution in hybrid 3D endoscopy. In: Deserno, T.M., Handels, H., Meinzer, H.-P., Tolxdorff, T. (eds.) Bildverarbeitung für die Medizin 2014. Informatik aktuell, pp. 84–89. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  12. 12.
    Kurmankhojayev, D., Hasler, N., Theobalt, C.: Monocular pose capture with a depth camera using a sums-of-gaussians body model. In: Weickert, J., Hein, M., Schiele, B. (eds.) GCPR 2013. LNCS, vol. 8142, pp. 415–424. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  13. 13.
    Liu, C.: Beyond pixels: exploring new representations and applications for motion analysis. Ph.D. thesis, Massachusetts Institute of Technology (2009)Google Scholar
  14. 14.
    Milanfar, P.: Super-Resolution Imaging. CRC Press, Boca Raton (2010)Google Scholar
  15. 15.
    Nabney, I.T.: NETLAB: Algorithms for Pattern Recognition. Advances in Pattern Recognition, 1st edn. Springer, Heidelberg (2002)Google Scholar
  16. 16.
    Park, J., Kim, H., Tai, Y., Brown, M., Kweon, I.: High quality depth map upsampling for 3D-TOF cameras. In: International Conference on Computer Vision, pp. 1623–1630 (2011)Google Scholar
  17. 17.
    Rajagopalan, A.N., Bhavsar, A., Wallhoff, F., Rigoll, G.: Resolution enhancement of PMD range maps. In: Rigoll, G. (ed.) DAGM 2008. LNCS, vol. 5096, pp. 304–313. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  18. 18.
    Schultz, R.R., Stevenson, R.L.: Extraction of high-resolution frames from video sequences. IEEE Trans. Image Process. 5, 996–1011 (1996)CrossRefGoogle Scholar
  19. 19.
    Schuon, S., Theobalt, C., Davis, J., Thrun, S.: High-quality scanning using time-of-flight depth superresolution. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 1–7 (2008)Google Scholar
  20. 20.
    Schuon, S., Theobalt, C., Davis, J., Thrun, S.: LidarBoost: depth superresolution for ToF 3D shape scanning. In: IEEE Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 343–350 (2009)Google Scholar
  21. 21.
    Schwarz, S., Sjostrom, M., Olsson, R.: A weighted optimization approach to time-of-flight sensor fusion. IEEE Trans. Image Process. 23(1), 214–225 (2014)CrossRefGoogle Scholar
  22. 22.
    Shotton, J., Girshick, R., Fitzgibbon, A., Sharp, T., Cook, M., Finocchio, M., Moore, R., Kohli, P., Criminisi, A., Kipman, A., Blake, A.: Efficient human pose estimation from single depth images. Pattern Anal. Mach. Intell. 35(12), 2821–2840 (2013)CrossRefGoogle Scholar
  23. 23.
    Wasza, J., Bauer, S., Haase, S., Schmid, M., Reichert, S., Hornegger, J.: RITK: the range imaging toolkit - a framework for 3-D range image stream processing. In: VMV, pp. 57–64. Eurographics Association (2011)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Florin C. Ghesu
    • 1
  • Thomas Köhler
    • 1
    • 2
    Email author
  • Sven Haase
    • 1
  • Joachim Hornegger
    • 1
    • 2
  1. 1.Pattern Recognition LabFriedrich-Alexander-Universität Erlangen-NürnbergErlangenGermany
  2. 2.Erlangen Graduate School in Advanced Optical Technologies (SAOT)ErlangenGermany

Personalised recommendations