A Spectral Approach for Probabilistic Grammatical Inference on Trees

  • Raphaël Bailly
  • Amaury Habrard
  • François Denis
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 6331)


We focus on the estimation of a probability distribution over a set of trees. We consider here the class of distributions computed by weighted automata - a strict generalization of probabilistic tree automata. This class of distributions (called rational distributions, or rational stochastic tree languages - RSTL) has an algebraic characterization: All the residuals (conditional) of such distributions lie in a finite-dimensional vector subspace. We propose a methodology based on Principal Components Analysis to identify this vector subspace. We provide an algorithm that computes an estimate of the target residuals vector subspace and builds a model which computes an estimate of the target distribution.


Hide Markov Model Rational Distribution Vector Subspace Kernel Principal Component Analysis Tree Series 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Carrasco, R., Oncina, J., Calera-Rubio, J.: Stochastic inference of regular tree languages. Machine Learning 44, 185–197 (2001)zbMATHCrossRefGoogle Scholar
  2. 2.
    Denis, F., Esposito, Y.: On rational stochastic languages. Fundamenta Informaticae 86, 41–77 (2008)zbMATHMathSciNetGoogle Scholar
  3. 3.
    Denis, F., Habrard, A.: Learning rational stochastic tree languages. In: Hutter, M., Servedio, R.A., Takimoto, E. (eds.) ALT 2007. LNCS (LNAI), vol. 4754, pp. 242–256. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  4. 4.
    Denis, F., Gilbert, E., Habrard, A., Ouardi, F., Tommasi, M.: Relevant representations for the inference of rational stochastic tree languages. In: Grammatical Inference: Algorithms and Applications, 9th International Colloquium, pp. 57–70. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  5. 5.
    Bailly, R., Denis, F., Ralaivola, L.: Grammatical inference as a principal component analysis problem. In: Proceedings of the 26th International Conference on Machine Learning, Montréal, Canada, pp. 33–40. Omnipress (2009)Google Scholar
  6. 6.
    Clark, A., Costa Florêncio, C., Watkins, C.: Languages as hyperplanes: grammatical inference with string kernels. In: Fürnkranz, J., Scheffer, T., Spiliopoulou, M. (eds.) ECML 2006. LNCS (LNAI), vol. 4212, pp. 90–101. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  7. 7.
    Hsu, D., Kakade, S., Zhang, T.: A spectral algorithm for learning hidden markov models. In: Proceedings of COLT 2009, Springer, Heidelberg (2009)Google Scholar
  8. 8.
    Hsu, D., Kakade, S., Zhang, T.: A spectral algorithm for learning hidden markov models. Technical report, Arxiv archive (2009),
  9. 9.
    Denis, F., Esposito, Y.: Learning classes of probabilistic automata. In: Shawe-Taylor, J., Singer, Y. (eds.) COLT 2004. LNCS (LNAI), vol. 3120, pp. 124–139. Springer, Heidelberg (2004)CrossRefGoogle Scholar
  10. 10.
    Comon, H., Dauchet, M., Gilleron, R., Jacquemard, F., Lugiez, D., Löding, C., Tison, S., Tommasi, M.: Tree automata techniques and applications (2007), (release October 12, 2007)
  11. 11.
    Berstel, J., Reutenauer, C.: Recognizable formal power series on trees. Theorical computer science 18, 115–148 (1982)zbMATHCrossRefMathSciNetGoogle Scholar
  12. 12.
    Högberg, J., Maletti, A., Vogler, H.: Bisimulation minimisation of weighted automata on unranked trees. Fundam. Inform. 92(1-2), 103–130 (2009)zbMATHGoogle Scholar
  13. 13.
    Borchardt, B.: The Theory of Recognizable Tree Series. PhD thesis, TU Dresden (2004)Google Scholar
  14. 14.
    McDiarmid, C.: On the method of bounded differences. In: Surveys in Combinatorics, pp. 148–188. Cambridge University Press, Cambridge (1989)Google Scholar
  15. 15.
    Stewart, G., Sun, J.G.: Matrix Perturbation Theory. Academic Press, London (1990)zbMATHGoogle Scholar
  16. 16.
    Zwald, L., Blanchard, G.: On the convergence of eigenspaces in kernel principal component analysis. In: Proceedings of NIPS 2005 (2006)Google Scholar
  17. 17.
    Shawe-Taylor, J., Cristianini, N., Kandola, J.: On the concentration of spectral properties. In: Proc. of NIPS, vol. 14, pp. 511–517. MIT Press, Cambridge (2001)Google Scholar
  18. 18.
    Smola, A., Gretton, A., Song, L., Schölkopf, B.: A hilbert space embedding for distributions. In: 18th International Conference on Algorithmic Learning Theory, pp. 13–31. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  19. 19.
    Song, L., Boots, B., Saddiqi, S., Gordon, G., Smola, A.: Hilbert space embeddings of Hidden Markov Models. In: Proceedings of ICML 2010 (2010)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2010

Authors and Affiliations

  • Raphaël Bailly
    • 1
  • Amaury Habrard
    • 1
  • François Denis
    • 1
  1. 1.Laboratoire d’Informatique Fondamentale de Marseille, UMR CNRS 6166Aix-Marseille Université CMIMarseille cedex 13France

Personalised recommendations