Advertisement

Regularization of Hidden Markov Models Embedded into Reproducing Kernel Hilbert Space

  • Galyna Kriukova
  • Mykola Glybovets
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 836)

Abstract

Hidden Markov models (HMMs) are well-known probabilistic graphical models for time series of discrete, partially observable stochastic processes. In this paper, we discuss an approach to extend the application of HMMs to non-Gaussian continuous distributions by embedding the belief about the state into a reproducing kernel Hilbert space (RKHS), and reduce tendency to overfitting and computational complexity of algorithm by means of various regularization techniques, specifically, Nyström subsampling. We investigate, theoretically and empirically, regularization and approximation bounds, the effectiveness of kernel samples as landmarks in the Nyström method for low-rank approximations of kernel matrices. Furthermore, we discuss applications of the method to real-world problems, comparing the approach to several state-of-the-art algorithms.

Keywords

Hidden Markov model Regularization Reproducing kernel Hilbert space Online algorithm 

Notes

Acknowledgment

Galyna Kriukova would like to thank Prof. Dr. Sergei Pereverzyev, Johann Radon Institute for Computational and Applied Mathematics (RICAM) of the Austrian Academy of Sciences, for sharing his wisdom and support.

References

  1. 1.
    Bargi, A., Xu, R.Y.D., Piccardi, M.: AdOn HDP-HMM: an adaptive online model for segmentation and classification of sequential data. IEEE Trans. Neural Netw. Learn. Syst. PP(99), 1–16 (2018).  https://doi.org/10.1109/TNNLS.2017.2742058CrossRefGoogle Scholar
  2. 2.
    Bauer, F., Pereverzev, S., Rosasco, L.: On regularization algorithms in learning theory. J. Complex. 23(1), 52–72 (2007).  https://doi.org/10.1016/j.jco.2006.07.001MathSciNetCrossRefMATHGoogle Scholar
  3. 3.
    Fong, S., Fiaidhi, J., Mohammed, S., Moutinho, L.: Real-time decision rules for diabetes therapy management by data stream mining. IT Prof. PP(99), 1 (2017).  https://doi.org/10.1109/MITP.2017.265104658CrossRefGoogle Scholar
  4. 4.
    Goh, C.Y., Dauwels, J., Mitrovic, N., Asif, M.T., Oran, A., Jaillet, P.: Online map-matching based on hidden Markov model for real-time traffic sensing applications. In: 2012 15th International IEEE Conference on Intelligent Transportation Systems, pp. 776–781 (2012).  https://doi.org/10.1109/ITSC.2012.6338627
  5. 5.
    Harchaoui, Z., Vallet, F., Lung-Yut-Fong, A., Cappe, O.: A regularized kernel-based approach to unsupervised audio segmentation. In: 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1665–1668 (2009).  https://doi.org/10.1109/ICASSP.2009.4959921
  6. 6.
    Jackson, Z., Souza, C., Flaks, J., Nicolas, H.: Jakobovski/free-spoken-digit-dataset v1.0.7 (2018).  https://doi.org/10.5281/zenodo.1136198
  7. 7.
    Kimeldorf, G., Wahba, G.: Some results on Tchebycheffian spline functions. J. Math. Anal. Appl. 33(1), 82–95 (1971).  https://doi.org/10.1016/0022-247X(71)90184-3MathSciNetCrossRefMATHGoogle Scholar
  8. 8.
    Kohlmorgen, J., Lemm, S.: A dynamic hmm for on–line segmentation of sequential data. In: Proceedings of the 14th International Conference on Neural Information Processing Systems: Natural and Synthetic, NIPS 2001, pp. 793–800. MIT Press, Cambridge (2001). http://dl.acm.org/citation.cfm?id=2980539.2980642
  9. 9.
    Krajzewicz, D., Erdmann, J., Behrisch, M., Bieker, L.: Recent development and applications of SUMO - simulation of urban MObility. Int. J. Adv. Syst. Meas. 5(3&4), 128–138 (2012)Google Scholar
  10. 10.
    Kriukova, G., Panasiuk, O., Pereverzyev, S.V., Tkachenko, P.: A linear functional strategy for regularized ranking. Neural Netw. 73, 26–35 (2016).  https://doi.org/10.1016/j.neunet.2015.08.012CrossRefGoogle Scholar
  11. 11.
    Kriukova, G., Pereverzyev, S., Tkachenko, P.: Nyström type subsampling analyzed as a regularized projection. Inverse Probl. 33(7), 074001 (2017). http://stacks.iop.org/0266-5611/33/i=7/a=074001CrossRefGoogle Scholar
  12. 12.
    Kriukova, G., Shvai, N., Pereverzyev, S.V.: Application of regularized ranking and collaborative filtering in predictive alarm algorithm for nocturnal hypoglycemia prevention. In: 2017 9th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), vol. 2, pp. 634–638 (2017).  https://doi.org/10.1109/IDAACS.2017.8095169
  13. 13.
    Lu, S., Pereverzev, S.V.: Regularization theory for ILL-posed problems: selected topics. Inverse and Ill-Posed Problems Series. De Gruyter, Berlin (2013). http://cds.cern.ch/record/1619978
  14. 14.
    Muandet, K., Fukumizu, K., Sriperumbudur, B., Schölkopf, B.: Kernel mean embedding of distributions: a review and beyond. Found. Trends Mach. Learn. 10(1–2), 1–141 (2017).  https://doi.org/10.1561/2200000060CrossRefMATHGoogle Scholar
  15. 15.
    Newson, P., Krumm, J.: Hidden Markov map matching through noise and sparseness. In: Proceedings of the 17th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, GIS 2009, pp. 336–343. ACM, New York (2009).  https://doi.org/10.1145/1653771.1653818
  16. 16.
    OpenStreetMap contributors: Planet dump retrieved from https://planet.osm.org (2017). https://www.openstreetmap.org
  17. 17.
    Smola, A., Gretton, A., Song, L., Scholkopf, B.: A hilbert space embedding for distributions. In: Hutter, M., Servedio, R.A., Takimoto, E. (eds.) Algorithmic Learning Theory, pp. 13–31. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  18. 18.
    Smola, A.J., Schlkopf, B.: Sparse greedy matrix approximation for machine learning, pp. 911–918. Morgan Kaufmann (2000)Google Scholar
  19. 19.
    Song, L., Boots, B., Siddiqi, S.M., Gordon, G., Smola, A.: Hilbert space embeddings of hidden Markov models. In: Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML 2010, pp. 991–998. Omnipress, Madison (2010). http://dl.acm.org/citation.cfm?id=3104322.3104448
  20. 20.
    Song, L., Huang, J., Smola, A., Fukumizu, K.: Hilbert space embeddings of conditional distributions with applications to dynamical systems. In: Proceedings of the 26th Annual International Conference on Machine Learning, ICML 2009, pp. 961–968. ACM, New York (2009).  https://doi.org/10.1145/1553374.1553497
  21. 21.
    Sudakov, O., Kriukova, G., Natarov, R., Gaidar, V., Maximyuk, O., Radchenko, S., Isaev, D.: Distributed system for sampling and analysis of electroencephalograms. In: 2017 9th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), vol. 1, pp. 306–310 (2017).  https://doi.org/10.1109/IDAACS.2017.8095095
  22. 22.
    Williams, C., Seeger, M.: Using the Nyström method to speed up kernel machines. In: Advances in Neural Information Processing Systems 13, pp. 682–688. MIT Press (2001)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.National University of Kyiv-Mohyla AcademyKyivUkraine

Personalised recommendations