Abstract
Adaptive classification evolving over time is an important learning task that arises in many applications. In this paper, a sequential dynamic multi-class model (SDMM) is proposed for representing the multi-class adaptive learning task, which is based on the polychotomous response model and dynamic logistic regression. Multiple state chains in the SDMM are coupled by the observable labels and feature vectors. Each state chain is modeled as a first-order Markov process with time-varying covariance parameters for characterizing the non-stationary generating process of sequential labels. Augmented auxiliary variables are introduced for developing efficient inference procedures according to the popular data augmentation strategy. Variational Bayesian methods are applied to estimate the dynamic state variables and augmented auxiliary variables recursively. According to the results of recursive filtering procedures using mean-field approximation forms, one-step-ahead predicted probabilities are calculated by marginalizing the state variables. Experiment results based on both synthetic and real data show that the proposed model significantly outperforms the non-sequential static methods for the multi-class adaptive learning problems with missing labels. Encouraging results have been obtained by comprising well-known multi-class data stream algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Millan, J.D.R.: On the need for on-line learning in brain-computer interfaces. In: Proceedings of the 23rd International Joint Conference on Neural Networks, pp. 2877–2882 (2004)
Shenoy, P., Krauledat, M., Blankertz, B., Rao, R.P.N., Muller, K.-R.: Towards adaptive classification for BCI. Journal of Neural Engineering 3, R13–R23 (2006)
Hojen-Sorensen P. A.d.F.R, Freitas N., Fog T.: On-line probabilistic classification with particle filters. In: Proceedings of the 2000 IEEE Signal Processing Society Workshop, pp. 386–395 (2000)
Penny, W.D., Roberts, S.J.: Dynamic logistic regression. In: International Joint Conference on Neural Network, pp. 1562–1567 (1999)
Sykacek, P., Roberts, S.J., Stokes, M.: Adaptive BCI based on variational Bayesian Kalman filtering: an empirical evaluation. IEEE Trans. Biomedical Engineering. 51(5), 719–727 (2004)
Lee, S.M., Roberts, S.J.: Sequential dynamic classification using latent variable models. Technical Report PARG 08-02, University of Oxford (2008)
Yoon, J.W., Roberts, S.J., Dyson, M., Gan, J.Q.: Sequential Bayesian estimation for adaptive classification. In: IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems, pp. 601–605 (2008)
Yoon, J.W., Roberts, S.J., Dyson, M., Gan, J.Q.: Adaptive classification for brain computer interface systems using sequential Monte Carlo sampling. Neural Networks 22(9), 1286–1294 (2009)
Lowne, D.R., Roberts, S.J., Garnett, R.: Sequential non-stationary dynamic classification with sparse feedback. Pattern Recognition 43(3), 897–905 (2010)
Kuncheva, L.I., Plumpton, C.O.: Adaptive learning rate for online linear discriminant classifiers. In: da Vitoria Lobo, N., Kasparis, T., Roli, F., Kwok, J.T., Georgiopoulos, M., Anagnostopoulos, G.C., Loog, M. (eds.) S+SSPR 2008. LNCS, vol. 5342, pp. 510–519. Springer, Heidelberg (2008)
Tomas, A.: A dynamic Logistic model for combining classifier outputs. DPhil thesis, Lady Margaret Hall, University of Oxford (2008)
Hulten, G., Spencer, L., Domingos, P.: Mining time-changing data streams. In: Proc. ACM SIGKDD 2001, pp. 97–106 (2001)
Sarkka, S., Nummenmaa, A.: Recursive noise adaptive Kalman filtering by variational Bayesian approximations. IEEE Trans. Automation Control. 54(3), 596–600 (2009)
Albert, J.H., Chib, S.: Bayesian analysis of binary and polychotomous response data. Journal of the American Statistical Association 88(422), 669–679 (1993)
Girolami, M., Rogers, S.: Variational Bayesian multinomial probit regression with Gaussian process priors. Neural Computation 18(8), 1790–1817 (2006)
Beal, M.J.: Variational algorithms for ap-proximation methods. Ph.D. dissertation, Gatsby Computational Neuroscience Unit, University College London (2003)
Schlogl, A.: The electroencephalogram and the adaptive autoregressive model: theory and applications. Shaker Verlag, Aachen (2000)
Hsu, C.W., Lin, C.J.: A comparison of methods for multi-class support vector machines. IEEE Transactions on Neural Networks 13(2), 415–425 (2002)
Blanchard, B., Mueller, K.R., Krusienski, D., Schalk, G., et al.: The BCI competition III: Validating alternative approaches to actual BCI problems. IEEE Trans. Neural Systems and Rehabilitation Engineering. 14(2), 153–159 (2006)
Schlogl, A., Lee, F., Bischof, H., Pfurtscheller, G.: Characterization of four-class motor imagery EEG data for the BCI-competition 2005. Journal of Neural Engineering 2, L14–L22 (2005)
Hashemi, A., Yang, Y., Mirzamomen, Z., Kangavari, M.: Adapted one-versus-all decision trees for data stream classification. IEEE Transactions on Knowledge and Data Engineering 21(5), 624–637 (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Qing, X., Wang, X. (2011). A Sequential Dynamic Multi-class Model and Recursive Filtering by Variational Bayesian Methods. In: Huang, J.Z., Cao, L., Srivastava, J. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2011. Lecture Notes in Computer Science(), vol 6634. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-20841-6_25
Download citation
DOI: https://doi.org/10.1007/978-3-642-20841-6_25
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-20840-9
Online ISBN: 978-3-642-20841-6
eBook Packages: Computer ScienceComputer Science (R0)