Randomly Sparsified Synthesis for Model-Based Deformation Analysis

  • Stefan ReinholdEmail author
  • Andreas Jordt
  • Reinhard Koch
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9796)


The tracking of deformation is one of the current challenges in computer vision. Analysis by Synthesis (AbS) based deformation tracking provides a way to fuse color and depth data into a single optimization problem very naturally. Previous work has shown that this can be done very efficiently using sparse synthesis. Although sparse synthesis allows AbS-based tracking to perform in real-time, it requires a great amount of problem specific customization and is limited to certain scenarios. This article introduces a new way of randomized adaptive sparsification of the reference model that adjusts the sparsification during the optimization process according to the required accuracy of the current optimization step. It will be shown that the efficiency of AbS can be increased significantly using the proposed method.


Input Image Triangle Mesh Deformation Space Order Change Depth Sensor 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Auger, A., Brockhoff, D., Hansen, N.: Benchmarking the (1,4)-CMA-ES with mirrored sampling and sequential selection on the noisy BBOB-2010 testbed. In: The 12th Annual Conference Companion on Genetic and Evolutionary Computation, pp. 1625–1632, July 2010Google Scholar
  2. 2.
    Besl, P.J., McKay, N.D.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(22), 239–256 (1992)CrossRefGoogle Scholar
  3. 3.
    Bregler, C., Hertzmann, A., Biermann, H.: Recovering non-rigid 3D shape from image streams. In: CVPR, vol. 2, pp. 2690–2696 (2000)Google Scholar
  4. 4.
    Cagniart, C., Boyer, E., Ilic, S.: Iterative mesh deformation for dense surface tracking. In: 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops, pp. 1465–1472, January 2009Google Scholar
  5. 5.
    Del Bue, A., Agapito, L.: Non-rigid stereo factorization. Int. J. Comput. Vis. 66(22), 193–207 (2006)Google Scholar
  6. 6.
    Fayad, J., Agapito, L., Del Bue, A.: Piecewise quadratic reconstruction of non-rigid surfaces from monocular sequences. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 297–310. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  7. 7.
    Gargallo, P., Prados, E., Sturm, P.: Minimizing the reprojection error in surface reconstruction from images. In: 2007 IEEE 11th International Conference on Computer Vision, pp. 1–8. IEEE (2007)Google Scholar
  8. 8.
    Hansen, N.: The CMA evolution strategy: a comparing review. In: Lozano, J.A., Larrañaga, P., Inza, I., Bengoetxea, E. (eds.) Towards a New Evolutionary Computation. Studies in Fuzziness and Soft Computing, vol. 192, pp. 75–102. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  9. 9.
    Hansen, N., Ostermeier, A.: Completely derandomized self-adaptation in evolution strategies. Evol. Comput. 9(2), 159–195 (2001)CrossRefGoogle Scholar
  10. 10.
    Helten, T., Baak, A., Bharaj, G., Müller, M., Seidel, H.P., Theobalt, C.: Personalization and evaluation of a real-time depth-based full body tracker. In: 3DV, pp. 279–286 (2013)Google Scholar
  11. 11.
    Innmann, M., Zollhöfer, M., Nießner, M., Theobalt, C., Stamminger, M.: VolumeDeform: real-time volumetric non-rigid reconstruction, January 2016. CoRR abs/1603.08161v2 [cs.CV]Google Scholar
  12. 12.
    Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R.A., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A.J., et al.: KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera. In: UIST, pp. 559–568, January 2011Google Scholar
  13. 13.
    Jordt, A., Koch, R.: Fast tracking of deformable objects in depth and colour video. In: BMVC, pp. 1–11, January 2011Google Scholar
  14. 14.
    Jordt, A., Koch, R.: Direct model-based tracking of 3D object deformations in depth and color video. Int. J. Comput. Vis. 102(11), 239–255 (2012)MathSciNetGoogle Scholar
  15. 15.
    Jordt, A., Koch, R.: Reconstruction of deformation from depth and color video with explicit noise models. In: Grzegorzek, M., Theobalt, C., Koch, R., Kolb, A. (eds.) Time-of-Flight and Depth Imaging. LNCS, vol. 8200, pp. 128–146. Springer, Heidelberg (2013)Google Scholar
  16. 16.
    Kambhamettu, C., Goldgof, D., He, M., Laskov, P.: 3D nonrigid motion analysis under small deformations. Image Vis. Comput. 21(33), 229–245 (2003)CrossRefGoogle Scholar
  17. 17.
    Li, H., Adams, B., Guibas, L.J., Pauly, M.: Robust single-view geometry and motion reconstruction. ACM Trans. Graph. 28(5), 175:1–175:10 (2009)CrossRefGoogle Scholar
  18. 18.
    Qian, C., Sun, X., Wei, Y., Tang, X., Sun, J.: Realtime and robust hand tracking from depth. In: CVPR, pp. 1106–1113, January 2014Google Scholar
  19. 19.
    Rosenhahn, B., Kersting, U.G., Powell, K., Klette, R., Klette, G., Seidel, H.P.: A system for articulated tracking incorporating a clothing model. Mach. Vis. Appl. 18(1), 25–40 (2007)CrossRefGoogle Scholar
  20. 20.
    Salzmann, M., Hartley, R.I., Fua, P.: Convex optimization for deformable surface 3-D tracking. In: ICCV, pp. 1–8, January 2007Google Scholar
  21. 21.
    Sharp, T., Keskin, C., Robertson, D., Taylor, J., Shotton, J.: Accurate, robust, and flexible real-time hand tracking. In: Proceedings of CHI 2015, January 2015Google Scholar
  22. 22.
    Shotton, J., Fitzgibbon, A.W., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., Blake, A.: Real-time human pose recognition in parts from single depth images. In: CVPR, pp. 1297–1304, January 2011Google Scholar
  23. 23.
    Steimle, J., Jordt, A., Maes, P.: Flexpad: highly flexible bending interactions for projected handheld displays. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 237–246, January 2013Google Scholar
  24. 24.
    Tomasi, C., Kanade, T.: Shape and motion from image streams under orthography: a factorization method. Int. J. Comput. Vis. 9(22), 137–154 (1992)CrossRefGoogle Scholar
  25. 25.
    Torresani, L., Hertzmann, A.: Learning non-rigid 3D shape from 2D motion. In: Advances in Neural Information Processing Systems, January 2003Google Scholar
  26. 26.
    Xu, W., Salzmann, M., Wang, Y., Liu, Y.: Nonrigid surface registration and completion from RGBD images. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part II. LNCS, vol. 8690, pp. 64–79. Springer, Heidelberg (2014)Google Scholar
  27. 27.
    Zollhöfer, M., Theobalt, C., Stamminger, M., Nießner, M., Izadi, S., Rehmann, C., Zach, C., Fisher, M., Wu, C., Fitzgibbon, A., et al.: Real-time non-rigid reconstruction using an RGB-D camera. ACM Trans. Graph. 33(44), 1–12 (2014)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Department of Computer ScienceUniversity of KielKielGermany

Personalised recommendations