Multimodal Nonlinear Filtering Using Gauss-Hermite Quadrature

  • Hannes P. Saal
  • Nicolas M. O. Heess
  • Sethu Vijayakumar
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6913)


In many filtering problems the exact posterior state distribution is not tractable and is therefore approximated using simpler parametric forms, such as single Gaussian distributions. In nonlinear filtering problems the posterior state distribution can, however, take complex shapes and even become multimodal so that single Gaussians are no longer sufficient. A standard solution to this problem is to use a bank of independent filters that individually represent the posterior with a single Gaussian and jointly form a mixture of Gaussians representation. Unfortunately, since the filters are optimized separately and interactions between the components consequently not taken into account, the resulting representation is typically poor. As an alternative we therefore propose to directly optimize the full approximating mixture distribution by minimizing the KL divergence to the true state posterior. For this purpose we describe a deterministic sampling approach that allows us to perform the intractable minimization approximately and at reasonable computational cost. We find that the proposed method models multimodal posterior distributions noticeably better than banks of independent filters even when the latter are allowed many more mixture components. We demonstrate the importance of accurately representing the posterior with a tractable number of components in an active learning scenario where we report faster convergence, both in terms of number of observations processed and in terms of computation time, and more reliable convergence on up to ten-dimensional problems.


Search Location Radial Basis Function Posterior Distribution Mixture Component Observation Function 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Abramowitz, M., Stegun, I.A.: Handbook of mathematical function with formulas, graphs, and mathematical tables. U.S. Dept. of Commerce (1972)Google Scholar
  2. 2.
    Alspach, D., Sorenson, H.: Nonlinear bayesian estimation using Gaussian sum approximations. IEEE Trans. Autom. Control 17(4), 439–448 (1972)CrossRefzbMATHGoogle Scholar
  3. 3.
    Arasaratnam, I., Haykin, S., Elliott, R.J.: Discrete-time nonlinear filtering algorithms using Gauss-Hermite quadrature. Proc. IEEE 95(5), 953–977 (2007)CrossRefGoogle Scholar
  4. 4.
    Bouchard, G., Zoeter, O.: Split variational inference. In: Proceedings of the 26th Annual International Conference on Machine Learning, ICML 2009, pp. 57–64. ACM, New York (2009)Google Scholar
  5. 5.
    Deisenroth, M.P., Huber, M.F., Hanebeck, U.D.: Analytic moment-based Gaussian process filtering. In: ICML (2009)Google Scholar
  6. 6.
    Girard, A., Rasmussen, C.E., Candela, J.Q., Murray-Smith, R.: Gaussian process priors with uncertain inputs—applications to multiple-step ahead time series forecasting. In: NIPS (2003)Google Scholar
  7. 7.
    Goldberger, J., Gordon, S., Greenspan, H.: An efficient image similarity measure based on approximations of KL-divergence between two Gaussian mixtures. In: Proc. IEEE Int. Conf. on Computer Vision (ICCV), vol. 1, pp. 487–493 (2003)Google Scholar
  8. 8.
    Hoffmann, G., Waslander, S., Tomlin, C.: Mutual information methods with particle filters for mobile sensor network control. In: Proc. IEEE Conf. on Decision and Control, pp. 1019–1024 (2006)Google Scholar
  9. 9.
    Huber, M.F., Bailey, T., Durrant-Whyte, H., Hanebeck, U.D.: On entropy approximation for Gaussian mixture random vectors. In: Proc. IEEE Int. Conf. on Multisensor Fusion and Integration for Intelligent Systems, pp. 181–188 (2008)Google Scholar
  10. 10.
    Ito, K., Xiong, K.: Gaussian filters for nonlinear filtering problems. IEEE Trans. Autom. Control. 45(5), 910–927 (2000)CrossRefzbMATHMathSciNetGoogle Scholar
  11. 11.
    Jaakola, T.S., Jordan, M.I.: Improving the mean field approximation via the use of mixture distributions. In: Jordan, M.I. (ed.) Learning in Graphical Models, pp. 163–173. Kluwer Academic Publishers, Dordrecht (1998)CrossRefGoogle Scholar
  12. 12.
    Julier, S., Uhlmann, J.: Unscented filtering and nonlinear estimation. Proc. IEEE 92(3), 401–422 (2004)CrossRefGoogle Scholar
  13. 13.
    Lawrence, N.D., Azzouzi, M.: A variational Bayesian committee of neural networks. Technical report, University of Cambridge, UK (1999)Google Scholar
  14. 14.
    Liu, Q., Pierce, D.A.: A note on Gauss-Hermite quadrature. Biometrika 81(3), 624–629 (1994)zbMATHMathSciNetGoogle Scholar
  15. 15.
    Park, J., Sandberg, I.W.: Universal approximation using radial-basis-function networks. Neural Comput. 3, 246–257 (1991)CrossRefGoogle Scholar
  16. 16.
    Saal, H.P., Ting, J., Vijayakumar, S.: Active sequential learning with tactile feedback. In: AISTATS (2010)Google Scholar
  17. 17.
    Torkkola, K.: Feature extraction by non-parametric mutual information maximization. J. Mach. Learn. Res. 3, 1415–1438 (2003)zbMATHMathSciNetGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2011

Authors and Affiliations

  • Hannes P. Saal
    • 1
  • Nicolas M. O. Heess
    • 1
  • Sethu Vijayakumar
    • 1
  1. 1.School of InformaticsUniversity of EdinburghEdinburghScotland, UK

Personalised recommendations