Apprentissage de Séquences Non-Indépendantes d’Exemples

  • Olivier Teytaud
Conference paper
Part of the Trends in Mathematics book series (TM)


Beaucoup de travaux récent considérent les applications pratiques des réseaux neuronaux (ou d’autres algorithmes proches) pour la modélisation de séries temporelles, par exemple chaotiques. Quelques papiers seulement (dont les résultats principaux sont rappelés ici) ont été consacrés aux applications de la partie théorique de l’apprentissage en la matière. Cet article fournit des rappels des résultats basés sur des propriétés d’ergodicité en matière d’apprentissage de suites non-indépendantes d’exemples, puis développe quelques nouveaux résultats.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    N.-T. Andersen, E. Giné, M. Ossiander, J. Zinn, The central limit theorem and the law of iterated logarithm for empirical processes under local conditions, Probability theory and related fields 77, 1988.Google Scholar
  2. [2]
    M.A. Arcones, Weak Convergence of stochastic processes indexed by smooth functions, Stochastic Processes and their Applications 62 115–138, 1996.MathSciNetzbMATHCrossRefGoogle Scholar
  3. [3]
    M.A. Arcones, B. Yu, Central limit theorems for empirical and U-processes of stationary mixing sequences. J. Theoret. Probab. 7 47–71, 1994.MathSciNetzbMATHCrossRefGoogle Scholar
  4. [4]
    R. Bowen, D. Ruelle, The ergodic theory of Axiom of A flows, invent. Math. 29, pp181–202,1975.MathSciNetzbMATHGoogle Scholar
  5. [5]
    K. Chellapilla, Evolving nonlinear controllers for backing up a Truckand-Trailer using evolutionary Programming, EPS 1998.Google Scholar
  6. [6]
    X. Chen, A. Guillin, The functional moderate deviations for Harris recurrent Markov chains and applications, preprint, Université Paris IX Dauphine, 2002.Google Scholar
  7. [7]
    P. Coullet, C. Tresser, Iterations d’endomorphismes et groupes de renormalisation, C.R. Acad. Sci. Paris 287, 577–580, 1978.MathSciNetzbMATHGoogle Scholar
  8. [8]
    S.-E. Decatur, PAC learning with Constant-Partition Classification Noise and Applications to Decision Tree induction, Proceedings of ICML 1997.Google Scholar
  9. [9]
    R. Devaney, Introduction to Chaotic Dynamical Systems: Theory and Experiments. Addison-Wesley, 1992.Google Scholar
  10. [10]
    L. Devroye, L. Gyorfi, G. Lugosi, A Probabilistic Theory of Pattern Recognition, Springer, 1996.Google Scholar
  11. [11]
    P. Diaconis, The cutoff phenomenon in finite Markov chains, Proc. Natl. Acad. Sci., USA, 93:1659–1664, 1996.MathSciNetzbMATHCrossRefGoogle Scholar
  12. [12]
    W. Doeblin, Eléments d’une théorie générale des chaînes simples constantes de Markov. Annales scientifiques de l’École Normale Supérieure 57 (III) 61–111, 1940.zbMATHGoogle Scholar
  13. [13]
    P. Doukhan, P. Massart, E. Rio, The functional central limit theorem for strongly mixing processes, Ann. Inst. Henri Poincare, 30, 63–82, 1994.MathSciNetzbMATHGoogle Scholar
  14. [14]
    R.-M. Dudley, Central limit theorems for empirical measures, Annals of probability 6, 1978. Correction: Annals of probability 7, 1978.Google Scholar
  15. [15]
    R.-M. Dudley, A course on empirical processes (Ecole d’été de Probabilité de Saint-Flour XII-1982), Lecture notes in Mathematics 1097, 2–141 (ed P.L. Hennequin), Springer-Verlag, New-York, 1984.Google Scholar
  16. [16]
    M. Feigenbaum, Qualitative universality for a class of nonlinear transformations, J. Stat. Phys. 19,25–52, 1978.MathSciNetzbMATHCrossRefGoogle Scholar
  17. [17]
    J.-M. Friedt, O. Teytaud, D. Gillet, M. Planat, Simultaneous amplitude and frequency noise analysis in Chua’s circuit - neural network based prediction and analysis, VIII Van Der Ziel Symposium on Quantum 1/f Noise and Other Low Frequency Fluctuations in Electronic Devices, 2000.Google Scholar
  18. [18]
    J.-M. Friedt, O. Teytaud, M. Planat, Learning from noise in Chua’s oscillator, accepted in ICNF, 2001.Google Scholar
  19. [19]
    D. Gamarnik, Extension of the PAC Framework to Finite and Countable Markov Chains, Proceedings of the Workshop on Computational Learning Theory, Morgan Kaufmann Publishers, 1999.Google Scholar
  20. [20]
    S. Hayashi, Connecting invariant manifolds and the solution of C1stability and a-stability conjectures for flows, Annals of Math. 145, 81–137, 1997.zbMATHCrossRefGoogle Scholar
  21. [21]
    S. Haykin, J.Principle, Using Neural Networks to Dynamically model Chaotic events such as sea clutter; making sense of a complex world, IEEE Signal Processing Magazine 66:81, 1998.Google Scholar
  22. [22]
    M. Hénon, A two-dimensional mapping with a strange attractor, Comm. Math. Phys. 50, 69–77, 1976.MathSciNetzbMATHCrossRefGoogle Scholar
  23. [23]
    R.A. Holmgren, A first course in discrete dynamical systems. Springer, 1996.CrossRefGoogle Scholar
  24. [24]
    W. Hong, The application of Radial Basis Function Networks to the prediction of Chaotic Time Series, Term Project, Course 2.160, Intelligent Control and Sensing, 1993.Google Scholar
  25. [25]
    T.A. Johansen, E. Weyer, On convergence Proofs in System Identification - A general principle using ideas from Learning Theory. Preprint, 1997: submitted to Elsevier Preprint.Google Scholar
  26. [26]
    V.-I. Kolcinski On the central limit theorem for empirical measures. Theory of probability and mathematical statistics 24, 1981.Google Scholar
  27. [27]
    H. Koul, Some convergence theorems for ranks and weighted empirical cumulatives. Annals of Mathematical Statistics 41, 1768–1773, 1970.MathSciNetzbMATHCrossRefGoogle Scholar
  28. [28]
    E.N. Lorenz, Deterministic nonperiodic flow, J. Atmosph. Sci. 20, 130–141, 1963.CrossRefGoogle Scholar
  29. [29]
    R. Mané, A proof of the C1-stability conjecture, publ. math. IHES 66 (161–210), 1988.zbMATHGoogle Scholar
  30. [30]
    E. Mammen, A.B. Tsybakov, Smooth discrimination analysis. Ann. Statist. 27, 1808–1829, 1999.MathSciNetzbMATHCrossRefGoogle Scholar
  31. [31]
    M. Marcus, J. Zinn, The bounded law of the iterated logarithm for the weighted empirical distribution in the non-iid case. Annals of Probability 12, 334–360, 1984.MathSciNetCrossRefGoogle Scholar
  32. [32]
    S.P. Meyn, R.L. Tweedie, Markov chains and stochastic stability. Springer-Verlag, 1993.zbMATHCrossRefGoogle Scholar
  33. [33]
    S.P. Meyn, R.L. Tweedie, Computable bounds for geometric convergence rates of Markov chains, Ann. Appl. Probab., 4(4):981–1011, 1994.MathSciNetzbMATHCrossRefGoogle Scholar
  34. [34]
    O. Michel, Notes sur la prédiction de séries temporelles issues de systèmes chaotiques, Ecole des Techniques Avancées en Signal-Image-Parole de Grenoble, Signal et non-linéaire. 1998.Google Scholar
  35. [35]
    M. Moonen, I. Proudler, An Introduction to Adaptive Signal Processing, HC63–64 Adaptive Signal Processing, Course Notes 1998–1999.Google Scholar
  36. [36]
    S. Mukherjee, E. Osuna, F. Girosi, Non-linear Prediction of Chaotic Time Series Using Support Vector Machines, Proc. of IEEE NNSP’97, Amelia Island, FL, 1997.Google Scholar
  37. [37]
    S. Newhouse, The abundance of wild hyperbolic sets and nonsmooth stable sets for diffeomorphisms, Publ. Math. IHES 50,101–151, 1979.MathSciNetzbMATHGoogle Scholar
  38. [38]
    M. Ossiander, A central limit theorem under metric entropy with L2 bracketing, Annals of probability 15, 1987.Google Scholar
  39. [39]
    D. Pollard, A central limit theorem for empirical processes. Journal of the australian mathematical society, A33, 1982.Google Scholar
  40. [40]
    L. Ralaivola, F. D’ALCHÉ-BUC, Incremental Support Vector Machine learning: a local approach. Accepted in Icann, 2001.Google Scholar
  41. [41]
    G.O. Roberts, J.S. Rosenthal, Geometric Ergodicity and Hybrid Markov chains, Electronic Communications in Probability, 2, 13–25, 1997.MathSciNetzbMATHCrossRefGoogle Scholar
  42. [42]
    J.S. Rosenthal, A review of asymptotic convergence for general state space Markov chains. Expository paper. Far East J. Theor. Stat. 5:37–50, 2001.zbMATHGoogle Scholar
  43. [43]
    D. Ruelle, A measure associated with Axiom A attractors, Amer. J. Math. 98, 619–654, 1976.MathSciNetzbMATHCrossRefGoogle Scholar
  44. [44]
    L. Saloff-Coste, Lectures on finite Markov chains. In Ecole d“et’e de probabilit’es de Saint-Flour XXVI, P. Bernard (ed.), L.N. in Math. 1664, 301–413. SpringerVerlag, New York, 1997.Google Scholar
  45. [45]
    G.R. Shorack, Convergence of reduced empirical and quantile processes with application to functions of order statistics in the non-iid case. Annals of Statistics 1, 146–152, 1973.MathSciNetzbMATHCrossRefGoogle Scholar
  46. [46]
    G.R. Shorack, The weighted empirical process of row independent random variables with arbitrary distribution functions. Statistica Neerlandica 33,169–189 (1980).MathSciNetCrossRefGoogle Scholar
  47. [47]
    G.R. Shorack, J.A. Wellner, Empirical Processes with applications to statistics, Wiley, New-York, 1986.Google Scholar
  48. [48]
    G.R. Shorack, J. Beirlant, The appropriate reduction of the weighted empirical process. Statistica Neerlandica 40, 123–12, 1986.MathSciNetzbMATHCrossRefGoogle Scholar
  49. [49]
    Y. Sinai, Gibbs measure in ergodic theory, Russian Math. Surveys 27, 21–69, 1972.MathSciNetzbMATHCrossRefGoogle Scholar
  50. [50]
    S. Smale, Differentiable dynamical systems, Bull. Amer. Math. Soc., 73, 747–817, 1967.MathSciNetzbMATHCrossRefGoogle Scholar
  51. [51]
    F. Takens, Detecting Strange Attractors in Turbulence, Lecture Notes in Math., 898:366–381, 1981.MathSciNetCrossRefGoogle Scholar
  52. [52]
    O. Teytaud, J.-M. Friedt, M. Planat, Neural network based prediction of Chua’s chaotic circuit frequency - possible extension to quartz oscillators, in proceedings of the EFTF, 2001Google Scholar
  53. [53]
    E. Trentin, M. Gori, Continuous Speech Recognition with a Robust Connectionist/Markovian Hybrid Model, in Proceedings of Icann 2001.Google Scholar
  54. [54]
    V.N. Vapnik, The Nature of Statistical Learning, Springer, 1995.zbMATHGoogle Scholar
  55. [55]
    M. Viana, Stochastic dynamics of deterministic systems
  56. [56]
    M. Vidyasagar, A theory of learning and generalization, Springer 1997.Google Scholar
  57. [57]
    A.-W. van der Vaart, J.-A. Wellner, Weak convergence and Empirical Processes, Springer, 1996.CrossRefGoogle Scholar
  58. [58]
    M. van Zujilen, Properties of the empirical distribution function for independent not identically distributed random variables. Annals of Probability 6, 250–266, 1978.CrossRefGoogle Scholar

Copyright information

© Springer Basel AG 2002

Authors and Affiliations

  • Olivier Teytaud
    • 1
  1. 1.ISCBron CedexFrance

Personalised recommendations