Joint Recursive Monocular Filtering of Camera Motion and Disparity Map

  • Johannes BergerEmail author
  • Christoph Schnörr
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9796)


Monocular scene reconstruction is essential for modern applications such as robotics or autonomous driving. Although stereo methods usually result in better accuracy than monocular methods, they are more expensive and more difficult to calibrate. In this work, we present a novel second order optimal minimum energy filter that jointly estimates the camera motion, the disparity map and also higher order kinematics recursively on a product Lie group containing a novel disparity group. This mathematical framework enables to cope with non-Euclidean state spaces, non-linear observations and high dimensions which is infeasible for most classical filters. To be robust against outliers, we use a generalized Charbonnier energy function in this framework rather than a quadratic energy function as proposed in related work. Experiments confirm that our method enables accurate reconstructions on-par with state-of-the-art.


Minimum energy filter Monocular reconstruction Camera motion estimation Lie groups 


  1. 1.
    Becker, F., Lenzen, F., Kappes, J.H., Schnörr, C.: Variational recursive joint estimation of dense scene structure and camera motion from monocular high speed traffic sequences. IJCV 105, 269–297 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Bellavia, F., Fanfani, M., Pazzaglia, F., Colombo, C.: Robust selective stereo SLAM without loop closure and bundle adjustment. In: Petrosino, A. (ed.) ICIAP 2013, Part I. LNCS, vol. 8156, pp. 462–471. Springer, Heidelberg (2013)CrossRefGoogle Scholar
  3. 3.
    Berger, J., Lenzen, F., Becker, F., Neufeld, A., Schnörr, C.: Second-Order Recursive Filtering on the Rigid-Motion Lie Group SE(3) Based on Nonlinear Observations (2015). ArXiv, preprint arXiv:1507.06810
  4. 4.
    Berger, J., Neufeld, A., Becker, F., Lenzen, F., Schnörr, C.: Second Order Minimum Energy Filtering on \({\text{ SE }}_{3}\) with Nonlinear Measurement Equations. In: Aujol, J.-F., Nikolova, M., Papadakis, N. (eds.) SSVM 2015. LNCS, vol. 9087, pp. 397–409. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-18461-6_32 Google Scholar
  5. 5.
    Bourmaud, G., Mégret, R.: Robust large scale monocular visual SLAM. In CVPR, pp. 1638–1647 (2015)Google Scholar
  6. 6.
    Bourmaud, G., Mégret, R., Arnaudon, M., Giremus, A.: Continuous-discrete extended Kalman filter on matrix lie groups using concentrated Gaussian distributions. J. Math. Imaging Vis. 51(1), 209–228 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Chikuse, Y.: Statistics on Special Manifolds, vol. 174. Springer, Heidelberg (2012)zbMATHGoogle Scholar
  8. 8.
    Daum, F., Huang, J.: Curse of dimensionality and particle filters. In: Aerospace Conference (2003)Google Scholar
  9. 9.
    Davison, A.J., Reid, I.D., Molton, N.D., Stasse, O.: MonoSLAM: real-time single camera SLAM. PAMI 29(6), 1052–1067 (2007)CrossRefGoogle Scholar
  10. 10.
    Dollár, P.: Piotr’s Computer Vision Matlab Toolbox (PMT).
  11. 11.
    Doucet, A., Freitas, N., Gordon, N.: An introduction to sequential Monte Carlo methods. In: Doucet, A., Freitas, N., Gordon, N. (eds.) Sequential Monte Carlo Methods in Practice, pp. 3–14. Springer, New York (2001)CrossRefGoogle Scholar
  12. 12.
    Engel, J., Schöps, T., Cremers, D.: LSD-SLAM: large-scale direct monocular SLAM. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014, Part II. LNCS, vol. 8690, pp. 834–849. Springer, Heidelberg (2014)Google Scholar
  13. 13.
    Engel, J., Sturm, J., Cremers, D.: Semi-dense visual odometry for a monocular camera. In: ICCV, pp. 1449–1456. IEEE (2013)Google Scholar
  14. 14.
    Frogerais, P., Bellanger, J., Senhadji, L.: Various ways to compute the continuous-discrete extended Kalman filter. IEEE Trans. Autom. Control 57, 1000–1004 (2012)MathSciNetCrossRefGoogle Scholar
  15. 15.
    Hairer, E., Lubich, C., Wanner, G.: Geometric Numerical Integration, vol. 31. Springer, Heidelberg (2006)zbMATHGoogle Scholar
  16. 16.
    Hirschmüller, H.: Stereo processing by semiglobal matching and mutual information. PAMI 30(2), 328–341 (2008)CrossRefGoogle Scholar
  17. 17.
    Kwon, J., Choi, M., Park, F.C., Chun, C.: Particle filtering on the Euclidean group: framework and applications. Robotica 25(6), 725–737 (2007)CrossRefGoogle Scholar
  18. 18.
    Mortensen, R.E.: Maximum-likelihood recursive nonlinear filtering. J. Opt. Theory Appl. 2(6), 386–394 (1968)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Neufeld, A., Berger, J., Becker, F., Lenzen, F., Schnörr, C.: Estimating vehicle ego-motion and piecewise planar scene structure from optical flow in a continuous framework. In: Gall, J., Gehler, P., Leibe, B. (eds.) GCPR 2015. LNCS, vol. 9358, pp. 41–52. Springer, Heidelberg (2015). doi: 10.1007/978-3-319-24947-6_4 CrossRefGoogle Scholar
  20. 20.
    Pizzoli, M., Forster, C., Scaramuzza, D.: REMODE: probabilistic, monocular dense reconstruction in real time. In: ICRA, pp. 2609–2616. IEEE (2014)Google Scholar
  21. 21.
    Pontryagin, L.S., Boltyanskii, V., Gamkrelidze, R., Mishchenko, E.: The Mathematical Theory of Optimal Processes. Interscience Publishers Inc., New York (1962)Google Scholar
  22. 22.
    Psota, E.T., Kowalczuk, J., Mittek, M., Perez, L.C.: MAP disparity estimation using hidden Markov trees. In: ICCV, pp. 2219–2227 (2015)Google Scholar
  23. 23.
    Revaud, J., Weinzaepfel, P., Harchaoui, Z., Schmid, C.: EpicFlow: edge-preserving interpolation of correspondences for optical flow. In: CVPR (2015)Google Scholar
  24. 24.
    Saccon, A., Trumpf, J., Mahony, R., Aguiar, A.P.: Second-order-optimal filters on lie groups. In: CDC (2013)Google Scholar
  25. 25.
    Saccon, A., Trumpf, J., Mahony, R., Aguiar, A.P.: Second-order-optimal minimum-energy filters on lie groups. IEEE TAC PP(99), 1 (2015)Google Scholar
  26. 26.
    Triggs, B., McLauchlan, P.F., Hartley, R.I., Fitzgibbon, A.W.: Bundle adjustment – a modern synthesis. In: Triggs, B., Zisserman, A., Szeliski, R. (eds.) ICCV-WS 1999. LNCS, vol. 1883, pp. 298–372. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  27. 27.
    Vogel, C., Schindler, K., Roth, S.: 3D scene flow estimation with a piecewise rigid scene model. IJCV 115(1), 1–28 (2015)MathSciNetCrossRefGoogle Scholar
  28. 28.
    Weinzaepfel, P., Revaud, J., Harchaoui, Z., Schmid, C.: Deepflow: large displacement optical flow with deep matching. In: ICCV, pp. 1385–1392 (2013)Google Scholar
  29. 29.
    Zamani, M., Trumpf, J., Mahoney, M.: A second order minimum-energy filter on the special orthogonal group. In: Proceedings of ACC (2012)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  1. 1.Image and Pattern Analysis GroupHeidelberg UniversityHeidelbergGermany

Personalised recommendations