# Unscented Kalman Filtering on Riemannian Manifolds

## Abstract

In recent years there has been a growing interest in problems, where either the observed data or hidden state variables are confined to a known Riemannian manifold. In sequential data analysis this interest has also been growing, but rather crude algorithms have been applied: either Monte Carlo filters or brute-force discretisations. These approaches scale poorly and clearly show a missing gap: no generic analogues to Kalman filters are currently available in non-Euclidean domains. In this paper, we remedy this issue by first generalising the unscented transform and then the unscented Kalman filter to Riemannian manifolds. As the Kalman filter can be viewed as an optimisation algorithm akin to the Gauss-Newton method, our algorithm also provides a general-purpose optimisation framework on manifolds. We illustrate the suggested method on synthetic data to study robustness and convergence, on a region tracking problem using covariance features, an articulated tracking problem, a mean value optimisation and a pose optimisation problem.

## Keywords

Riemannian manifolds Unscented Kalman filter Filtering theory Optimisation on manifolds## Notes

### Acknowledgements

The authors would like to thank the anonymous reviewers for detailed comments, which substantially improved the quality of the manuscript. Furthermore, Søren Hauberg would like to thank the Villum Foundation for financial support.

## Supplementary material

(MP4 1.4 MB)

(MP4 4.6 MB)

(MP4 4.7 MB)

(MP4 899 kB)

(MP4 15.0 MB)

(MP4 4.8 MB)

(MP4 4.7 MB)

## References

- 1.Balan, A.O., Sigal, L., Black, M.J.: A quantitative evaluation of video-based 3D person tracking. In: 2nd Joint IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, pp. 349–356 (2005) CrossRefGoogle Scholar
- 2.Bandouch, J., Engstler, F., Beetz, M.: Accurate human motion capture using an ergonomics-based anthropometric human model. In: AMDO’08: Proceedings of the 5th International Conference on Articulated Motion and Deformable Objects. Lecture Notes in Computer Science, vol. 5098, pp. 248–258. Springer, Berlin (2008) CrossRefGoogle Scholar
- 3.Bell, B.M., Cathey, F.W.: The iterated Kalman filter update as a Gauss-Newton method. IEEE Trans. Autom. Control
**38**, 294–297 (1993) MathSciNetMATHCrossRefGoogle Scholar - 4.Cappé, O., Godsill, S.J., Moulines, E.: An overview of existing methods and recent advances in sequential Monte Carlo. Proc. IEEE
**95**(5), 899–924 (2007) CrossRefGoogle Scholar - 5.do Carmo, M.P.: Riemannian Geometry. Birkhäuser, Boston (1992) MATHCrossRefGoogle Scholar
- 6.Caselles, V., Kimmel, R., Sapiro, G.: Geodesic active contours. Int. J. Comput. Vis.
**22**, 61–79 (1997) MATHCrossRefGoogle Scholar - 7.Engell-Nørregård, M., Erleben, K.: A projected back-tracking line-search for constrained interactive inverse kinematics. Comput. Graph.
**35**(2), 288–298 (2011) CrossRefGoogle Scholar - 8.Erleben, K., Sporring, J., Henriksen, K., Dohlmann, H.: Physics Based Animation. Charles River Media, Newton Center (2005) Google Scholar
- 9.Fletcher, P.T., Joshi, S.: Riemannian geometry for the statistical analysis of diffusion tensor data. Signal Process.
**87**, 250–262 (2007) MATHCrossRefGoogle Scholar - 10.Fletcher, P.T., Lu, C., Pizer, S.M., Joshi, S.: Principal geodesic analysis for the study of nonlinear statistics of shape. IEEE Trans. Med. Imaging
**23**(8), 995–1005 (2004) CrossRefGoogle Scholar - 11.Hairer, E., Lubich, C., Wanner, G.: Geometric Numerical Integration: Structure Preserving Algorithms for Ordinary Differential Equations. Springer, Berlin (2004) Google Scholar
- 12.Hauberg, S., Pedersen, K.S.: Stick it! Articulated tracking using spatial rigid object priors. In: Asian Conference on Computer Vision. Lecture Notes in Computer Science, vol. 6494. Springer, Berlin (2010) Google Scholar
- 13.Hauberg, S., Pedersen, K.S.: Predicting articulated human motion from spatial processes. Int. J. Comput. Vis.
**94**, 317–334 (2011) MathSciNetMATHCrossRefGoogle Scholar - 14.Hauberg, S., Pedersen, K.S.: HUMIM software for articulated tracking. Tech. Rep. 01/2012, Department of Computer Science, University of Copenhagen (2012) Google Scholar
- 15.Hauberg, S., Sommer, S., Pedersen, K.S.: Gaussian-like spatial priors for articulated tracking. In: ECCV. Lecture Notes in Computer Science, vol. 6311, pp. 425–437. Springer, Berlin (2010) Google Scholar
- 16.Hauberg, S., Sommer, S., Pedersen, K.S.: Natural metrics and least-committed priors for articulated tracking. Image Vis. Comput.
**30**(6–7), 453–461 (2012) CrossRefGoogle Scholar - 17.Julier, S.J., Uhlmann, J.K.: A new extension of the Kalman filter to nonlinear systems. In: International Symposium Aerospace/Defense Sensing, Simulation and Controls, pp. 182–193 (1997) Google Scholar
- 18.Kalman, R.: A new approach to linear filtering and prediction problems. J. Basic Eng.
**82**(D), 35–45 (1960) CrossRefGoogle Scholar - 19.Karcher, H.: Riemannian center of mass and mollifier smoothing. Commun. Pure Appl. Math.
**30**(5), 509–541 (1977) MathSciNetMATHCrossRefGoogle Scholar - 20.Kendall, D.G.: Shape manifolds, procrustean metrics, and complex projective spaces. Bull. Lond. Math. Soc.
**16**(2), 81–121 (1984) MathSciNetMATHCrossRefGoogle Scholar - 21.Kjellström, H., Kragić, D., Black, M.J.: Tracking people interacting with objects. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 747–754 (2010) Google Scholar
- 22.Kraft, E.: A quaternion-based unscented Kalman filter for orientation tracking. In: Proceedings of the Sixth International Conference on Information Fusion, pp. 47–54 (2003) Google Scholar
- 23.Kwon, J., Lee, K.M.: Monocular SLAM with locally planar landmarks via geometric rao-blackwellized particle filtering on Lie groups. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1522–1529 (2010) Google Scholar
- 24.Kwon, J., Lee, K.M., Park, F.C.: Visual tracking via geometric particle filtering on the affine group with optimal importance functions. In: Computer Vision and Pattern Recognition, pp. 991–998 (2009) Google Scholar
- 25.Lewis, F.L.: Optimal Estimation: With an Introduction to Stochastic Control Theory. Wiley, New York (1986) MATHGoogle Scholar
- 26.Li, R., Chellappa, R.: Aligning spatio-temporal signals on a special manifold. In: ECCV. Lecture Notes in Computer Science, vol. 6315, pp. 547–560. Springer, Berlin (2010) Google Scholar
- 27.Liu, X., Srivastava, A., Gallivan, K.: Optimal linear representations of images for object recognition. IEEE Trans. Pattern Anal. Mach. Intell.
**26**(5), 662–666 (2004) CrossRefGoogle Scholar - 28.van der Merwe, R., Doucet, A., Freitas, N.D., Wan, E.: The unscented particle filter. In: Advances in Neural Information Processing Systems (NIPS 2000), vol. 13, pp. 584–590. MIT Press, Cambridge (2001) Google Scholar
- 29.Misner, C., Thorne, K., Wheeler, J.: Gravitation. W.H. Freeman, New York (1973) Google Scholar
- 30.Pennec, X.: Intrinsic statistics on Riemannian manifolds: basic tools for geometric measurements. J. Math. Imaging Vis.
**25**(1), 127–154 (2006) MathSciNetCrossRefGoogle Scholar - 31.Pennec, X., Fillard, P., Ayache, N.: A Riemannian framework for tensor computing. Int. J. Comput. Vis.
**66**, 41–66 (2004) CrossRefGoogle Scholar - 32.Poppe, R.: Vision-based human motion analysis: an overview. Comput. Vis. Image Underst.
**108**(1–2), 4–18 (2007) CrossRefGoogle Scholar - 33.Porikli, F., Tuzel, O., Meer, P.: Covariance tracking using model update based on Lie algebra. In: Computer Vision and Pattern Recognition, vol. 1, pp. 728–735 (2006) Google Scholar
- 34.Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proc. IEEE
**77**(2), 257–286 (1989) CrossRefGoogle Scholar - 35.Sidenbladh, H., Black, M.J., Fleet, D.J.: Stochastic tracking of 3D human figures using 2D image motion. In: ECCV, vol. II. Lecture Notes in Computer Science, vol. 1843, pp. 702–718. Springer, Berlin (2000) Google Scholar
- 36.Sigal, L., Black, M.J.: HumanEva: synchronized video and motion capture dataset for evaluation of articulated human motion. Tech. Rep. CS-06-08, Brown University (2007) Google Scholar
- 37.Singhal, S., Wu, L.: Training multilayer perceptrons with the extended Kalman algorithm. In: Advances in Neural Information Processing Systems, vol. 1, pp. 133–140 (1989) Google Scholar
- 38.Sipos, B.J.: Application of the Manifold-Constrained unscented Kalman filter. In: Position, Location and Navigation Symposium, IEEE/ION, pp. 30–43 (2008) Google Scholar
- 39.Sommer, S., Tatu, A., Chen, C., Jørgensen, D.R., de Bruijne, M., Loog, M., Nielsen, M., Lauze, F.: Bicycle chain shape models. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 157–163. IEEE Computer Society, Los Alamitos (2009) CrossRefGoogle Scholar
- 40.Sommer, S., Lauze, F., Nielsen, M.: The differential of the exponential map, Jacobi fields and exact principal geodesic analysis. CoRR (2010). arXiv:1008.1902
- 41.Srivastava, A., Klassen, E.: Bayesian and geometric subspace tracking. Adv. Appl. Probab.
**36**(1), 43–56 (2004) MathSciNetMATHCrossRefGoogle Scholar - 42.Subbarao, R., Meer, P.: Nonlinear mean shift over Riemannian manifolds. Int. J. Comput. Vis.
**84**(1), 1–20 (2009) CrossRefGoogle Scholar - 43.Tidefelt, H., Schön, T.B.: Robust point-mass filters on manifolds. In: Proceedings of the 15th IFAC Symposium on System Identification (SYSID), pp. 540–545 (2009) Google Scholar
- 44.Tuzel, O., Porikli, F., Meer, P.: Region covariance: A fast descriptor for detection and classification. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV. Lecture Notes in Computer Science, vol. 3952, pp. 589–600. Springer, Berlin/Heidelberg (2006) Google Scholar
- 45.Tyagi, A., Davis, J.W.: A recursive filter for linear systems on Riemannian manifolds. In: Computer Vision and Pattern Recognition, pp. 1–8 (2008) Google Scholar
- 46.Wan, E.A., van der Merwe, R.: The unscented Kalman filter for nonlinear estimation. In: Adaptive Systems for Signal Processing, Communications, and Control Symposium, IEEE, pp. 153–158 (2002) Google Scholar
- 47.Ward, R.C.: Numerical computation of the matrix exponential with accuracy estimate. SIAM J. Numer. Anal.
**14**, 600–610 (1977) MathSciNetMATHCrossRefGoogle Scholar - 48.Wu, Y., Wu, B., Liu, J., Lu, H.: Probabilistic tracking on Riemannian manifolds. In: International Conference on Pattern Recognition, pp. 1–4 (2008) Google Scholar