Abstract
Hidden Markov models (HMMs) are well-known probabilistic graphical models for time series of discrete, partially observable stochastic processes. In this paper, we discuss an approach to extend the application of HMMs to non-Gaussian continuous distributions by embedding the belief about the state into a reproducing kernel Hilbert space (RKHS), and reduce tendency to overfitting and computational complexity of algorithm by means of various regularization techniques, specifically, Nyström subsampling. We investigate, theoretically and empirically, regularization and approximation bounds, the effectiveness of kernel samples as landmarks in the Nyström method for low-rank approximations of kernel matrices. Furthermore, we discuss applications of the method to real-world problems, comparing the approach to several state-of-the-art algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bargi, A., Xu, R.Y.D., Piccardi, M.: AdOn HDP-HMM: an adaptive online model for segmentation and classification of sequential data. IEEE Trans. Neural Netw. Learn. Syst. PP(99), 1–16 (2018). https://doi.org/10.1109/TNNLS.2017.2742058
Bauer, F., Pereverzev, S., Rosasco, L.: On regularization algorithms in learning theory. J. Complex. 23(1), 52–72 (2007). https://doi.org/10.1016/j.jco.2006.07.001
Fong, S., Fiaidhi, J., Mohammed, S., Moutinho, L.: Real-time decision rules for diabetes therapy management by data stream mining. IT Prof. PP(99), 1 (2017). https://doi.org/10.1109/MITP.2017.265104658
Goh, C.Y., Dauwels, J., Mitrovic, N., Asif, M.T., Oran, A., Jaillet, P.: Online map-matching based on hidden Markov model for real-time traffic sensing applications. In: 2012 15th International IEEE Conference on Intelligent Transportation Systems, pp. 776–781 (2012). https://doi.org/10.1109/ITSC.2012.6338627
Harchaoui, Z., Vallet, F., Lung-Yut-Fong, A., Cappe, O.: A regularized kernel-based approach to unsupervised audio segmentation. In: 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 1665–1668 (2009). https://doi.org/10.1109/ICASSP.2009.4959921
Jackson, Z., Souza, C., Flaks, J., Nicolas, H.: Jakobovski/free-spoken-digit-dataset v1.0.7 (2018). https://doi.org/10.5281/zenodo.1136198
Kimeldorf, G., Wahba, G.: Some results on Tchebycheffian spline functions. J. Math. Anal. Appl. 33(1), 82–95 (1971). https://doi.org/10.1016/0022-247X(71)90184-3
Kohlmorgen, J., Lemm, S.: A dynamic hmm for on–line segmentation of sequential data. In: Proceedings of the 14th International Conference on Neural Information Processing Systems: Natural and Synthetic, NIPS 2001, pp. 793–800. MIT Press, Cambridge (2001). http://dl.acm.org/citation.cfm?id=2980539.2980642
Krajzewicz, D., Erdmann, J., Behrisch, M., Bieker, L.: Recent development and applications of SUMO - simulation of urban MObility. Int. J. Adv. Syst. Meas. 5(3&4), 128–138 (2012)
Kriukova, G., Panasiuk, O., Pereverzyev, S.V., Tkachenko, P.: A linear functional strategy for regularized ranking. Neural Netw. 73, 26–35 (2016). https://doi.org/10.1016/j.neunet.2015.08.012
Kriukova, G., Pereverzyev, S., Tkachenko, P.: Nyström type subsampling analyzed as a regularized projection. Inverse Probl. 33(7), 074001 (2017). http://stacks.iop.org/0266-5611/33/i=7/a=074001
Kriukova, G., Shvai, N., Pereverzyev, S.V.: Application of regularized ranking and collaborative filtering in predictive alarm algorithm for nocturnal hypoglycemia prevention. In: 2017 9th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), vol. 2, pp. 634–638 (2017). https://doi.org/10.1109/IDAACS.2017.8095169
Lu, S., Pereverzev, S.V.: Regularization theory for ILL-posed problems: selected topics. Inverse and Ill-Posed Problems Series. De Gruyter, Berlin (2013). http://cds.cern.ch/record/1619978
Muandet, K., Fukumizu, K., Sriperumbudur, B., Schölkopf, B.: Kernel mean embedding of distributions: a review and beyond. Found. Trends Mach. Learn. 10(1–2), 1–141 (2017). https://doi.org/10.1561/2200000060
Newson, P., Krumm, J.: Hidden Markov map matching through noise and sparseness. In: Proceedings of the 17th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, GIS 2009, pp. 336–343. ACM, New York (2009). https://doi.org/10.1145/1653771.1653818
OpenStreetMap contributors: Planet dump retrieved from https://planet.osm.org (2017). https://www.openstreetmap.org
Smola, A., Gretton, A., Song, L., Scholkopf, B.: A hilbert space embedding for distributions. In: Hutter, M., Servedio, R.A., Takimoto, E. (eds.) Algorithmic Learning Theory, pp. 13–31. Springer, Heidelberg (2007)
Smola, A.J., Schlkopf, B.: Sparse greedy matrix approximation for machine learning, pp. 911–918. Morgan Kaufmann (2000)
Song, L., Boots, B., Siddiqi, S.M., Gordon, G., Smola, A.: Hilbert space embeddings of hidden Markov models. In: Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML 2010, pp. 991–998. Omnipress, Madison (2010). http://dl.acm.org/citation.cfm?id=3104322.3104448
Song, L., Huang, J., Smola, A., Fukumizu, K.: Hilbert space embeddings of conditional distributions with applications to dynamical systems. In: Proceedings of the 26th Annual International Conference on Machine Learning, ICML 2009, pp. 961–968. ACM, New York (2009). https://doi.org/10.1145/1553374.1553497
Sudakov, O., Kriukova, G., Natarov, R., Gaidar, V., Maximyuk, O., Radchenko, S., Isaev, D.: Distributed system for sampling and analysis of electroencephalograms. In: 2017 9th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), vol. 1, pp. 306–310 (2017). https://doi.org/10.1109/IDAACS.2017.8095095
Williams, C., Seeger, M.: Using the Nyström method to speed up kernel machines. In: Advances in Neural Information Processing Systems 13, pp. 682–688. MIT Press (2001)
Acknowledgment
Galyna Kriukova would like to thank Prof. Dr. Sergei Pereverzyev, Johann Radon Institute for Computational and Applied Mathematics (RICAM) of the Austrian Academy of Sciences, for sharing his wisdom and support.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Kriukova, G., Glybovets, M. (2019). Regularization of Hidden Markov Models Embedded into Reproducing Kernel Hilbert Space. In: Chertov, O., Mylovanov, T., Kondratenko, Y., Kacprzyk, J., Kreinovich, V., Stefanuk, V. (eds) Recent Developments in Data Science and Intelligent Analysis of Information. ICDSIAI 2018. Advances in Intelligent Systems and Computing, vol 836. Springer, Cham. https://doi.org/10.1007/978-3-319-97885-7_33
Download citation
DOI: https://doi.org/10.1007/978-3-319-97885-7_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-97884-0
Online ISBN: 978-3-319-97885-7
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)