A New Generative Feature Set Based on Entropy Distance for Discriminative Classification

  • Alessandro Perina
  • Marco Cristani
  • Umberto Castellani
  • Vittorio Murino
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5716)


Score functions induced by generative models extract fixed-dimensions feature vectors from different-length data observations by subsuming the process of data generation, projecting them in highly informative spaces called score spaces. In this way, standard discriminative classifiers such as support vector machines, or logistic regressors are proved to achieve higher performances than a solely generative or discriminative approach. In this paper, we present a novel score space that capture the generative process encoding it in an entropic feature vector. In this way, both uncertainty in the generative model learning step and “local” compliance of data observations with respect to the generative process can be represented. The proposed score space is presented for hidden Markov models and mixture of gaussian and is experimentally validated on standard benchmark datasets; moreover it can be applied to any generative model. Results show how it achieves compelling classification accuracies.


Support Vector Machine Hide Markov Model Discriminative Method Score Space Variable Length Sequence 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Tsuda, K., Kawanabe, M., Rätsch, G., Sonnenburg, S., Müller, K.-R.: A new discriminative kernel from probabilistic models. Neural Comput. 14(10), 2397–2414 (2002)CrossRefzbMATHGoogle Scholar
  2. 2.
    Jaakkola, T., Haussler, D.: Exploiting generative models in discriminative classifiers. In: NIPS (1998)Google Scholar
  3. 3.
    Smith, N., Gales, M.: Using svms to classify variable length speech patterns. Technical Report CUED/F-INGENF/TR.412, University of Cambridge, UK (2002)Google Scholar
  4. 4.
    Holub, A.D., Welling, M., Perona, P.: Combining generative models and fisher kernels for object recognition. In: ICCV, vol. 1, pp. 136–143 (2005)Google Scholar
  5. 5.
    Ng, A.Y., Jordan, M.I.: On discriminative vs. generative classifiers: A comparison of logistic regression and naive bayes. In: Dietterich, T.G., Becker, S., Ghahramani, Z. (eds.) NIPS. MIT Press, Cambridge (2002)Google Scholar
  6. 6.
    Rabiner, L.R.: A tutorial on Hidden Markov Models and selected applications in speech recognition. Proc. of IEEE 77(2), 257–286 (1989)CrossRefGoogle Scholar
  7. 7.
    Liang, P., Jordan, M.I.: An asymptotic analysis of generative, discriminative, and pseudolikelihood estimators. In: ICML 2008: Proceedings of the 25th international conference on Machine learning, pp. 584–591. ACM, New York (2008)CrossRefGoogle Scholar
  8. 8.
    Schölkopf, B., Smola, A.J.: Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. The MIT Press, Cambridge (2001)Google Scholar
  9. 9.
    Akhloufi, M.A., Larbi, W.B., Maldague, X.: Framework for color-texture classification in machine vision inspection of industrial products. In: ISIC, pp. 1067–1071 (2007)Google Scholar
  10. 10.
    Asuncion, A., Newman, D.J.: UCI machine learning repository (2007)Google Scholar
  11. 11.
    Mollineda, R.A., Vidal, E., Casacuberta, F.: Cyclic sequence alignments: Approximate versus optimal techniques. International Journal of Pattern Recognition and Artificial Intelligence 16, 291–299 (2002)CrossRefGoogle Scholar
  12. 12.
    Bicego, M., Trudda, A.: 2d shape classification using multifractional brownian motion. In: SSPR/SPR, pp. 906–916 (2008)Google Scholar
  13. 13.
    Michel, N., Horst, B.: Edit distance-based kernel functions for structural pattern classification. Pattern Recogn. 39(10), 1852–1863 (2006)CrossRefzbMATHGoogle Scholar
  14. 14.
    Williams, B.H., Toussaint, M., Storkey, A.J.: Extracting motion primitives from natural handwriting data. In: Kollias, S.D., Stafylopatis, A., Duch, W., Oja, E. (eds.) ICANN 2006. LNCS, vol. 4132, pp. 634–643. Springer, Heidelberg (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Alessandro Perina
    • 1
  • Marco Cristani
    • 1
    • 2
  • Umberto Castellani
    • 1
  • Vittorio Murino
    • 1
    • 2
  1. 1.Dipartimento di InformaticaUniversità degli Studi di VeronaVeronaItalia
  2. 2.Istituto Italiano di TecnologiaGenovaItalia

Personalised recommendations