Advertisement

Chinese Science Bulletin

, Volume 44, Issue 10, pp 865–880 | Cite as

Computational intelligence: From mathematical point of view

  • Minping Qian
  • Guanglu Gong
Reviews
  • 35 Downloads

Abstract

A simple but illustrative survey is given on various approaches of computational intelligence with their features, applications and the mathematical tools involved, among which the simulated annealing, neural networks, genetic and evolutionary programming, self-organizing learning and adapting algorithms, hidden Markov models are recommended intensively. The common mathematical features of various computational intelligence algorithms are exploited. Finally, two common principles of concessive strategies implicated in many computational intelligence algorithms are discussed.

Keywords

computational Intelligence simulated annealing neural network genetic algorithm evolutionary programming self-organizing learning self-adapting algorithm hidden Markov model concession strategy 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Li, G. J., Computational intelligence: An important field of research,Fundamental Study of Intelligent Computer ’94 (eds. Li, W., Huai, J. P., Bai, S.) (in Chinese), Beijing: Press of Tsinghua University, 1994, 9–12.Google Scholar
  2. 2.
    Dai, R. W., Semantics and grammar recognition based on artificial neural networks,Fundamental Study of Intelligent Computer’ 94 (Li, W., Huai, J. P., Bai, S.) (in Chinese), Beijing: Press of Tsinghua University, 1994, 1–5.Google Scholar
  3. 3.
    Sejnowski, T. J., Rosenberg, C. R., Parallel networks that learn pronounce English text,Complex Systems, 1987, (1): 145.Google Scholar
  4. 4.
    Fogel, L. J., Owens, A. J., Walsh, M. J.,Artificial Intelligence Through Simulated Evolution, New York: John Wiley, 1966.Google Scholar
  5. 5.
    Holland, J. H., Genetic algorithm and the optimal allocations of trials,SIAM J. of Computing, 1973, 2(2): 88.CrossRefGoogle Scholar
  6. 6.
    Kirkpatrick, S., Gelatt, C. D., Jr., Vecchi, M. P., Optimization by simulated annealing,IBM Research Report, 1982, Re 9353.Google Scholar
  7. 7.
    Macready, W. G., Siapas, A. G., Kauffman, S. A., Crilicality and parallelism in combinatorial optimization,Science, 1996, 271(5 Jan.): 56.PubMedCrossRefGoogle Scholar
  8. 8.
    Khas’ minskii, R. Z., Application of random noise to optimization and recognition problems,Problems of Information Transmission, 1965, 1(3): 89.Google Scholar
  9. 9.
    van Laavhoven, P. J. M., Aarts, E. H. Y. L.,Simulation Annealing: Theory and Applications, Holland: D Reidel Publishing Company, 1987.Google Scholar
  10. 10.
    Holly, R., Stroock, D., Simulated annealing via Sobolev inequalities,Commun. Math. Phy., 1988, 115: 553.CrossRefGoogle Scholar
  11. 11.
    Chiang, T. S., Chow, Y., On the convergence rate of annealing Process,SAIM J. Control Optim., 1988, 26: 1455.CrossRefGoogle Scholar
  12. 12.
    Chiang, T. S., Chow, Y., A limit theory for a class of inhomogeneous Markov processes, Ann.of Probability, 1989, 17: 1483.CrossRefGoogle Scholar
  13. 13.
    Hwang, C. R., Sheu, S. J., Large-time behavior of perturbed diffusion Markov processes with application to the second eigenvalue problem for Fokker-Plank operators and simulated annealing,Acta Applicae Math., 1990, 19: 253.CrossRefGoogle Scholar
  14. 14.
    Chiang, T. S., Sheu, S. J., Diffusions for global optimization in Ad,SIAM J. Control Optm., 1987, 25: 737.CrossRefGoogle Scholar
  15. 15.
    Fang, H., Gong, G., Qian, M. P., Annealing of iterative stochastic schemes,SIAM J. Control Optim., 1997, 35(8): 1886.CrossRefGoogle Scholar
  16. 16.
    Szu, H. H., Hartley, H. L., Fast simulated annealing,Phy. lett. A, 1987, 123(3-4): 157.CrossRefGoogle Scholar
  17. 17.
    Szu, H. H., Hartley, H. L., Non-convex optimization by fast simulated annealing,Proc. of IEEE, 1987, 75(11): 1538.CrossRefGoogle Scholar
  18. 18.
    Fang, H., Gong, G., Qian, M. P., Disconvergence of Cauchy annealing,Science in China, Ser. A, 1996, 39(9): 945.Google Scholar
  19. 19.
    Li, Y., Qian, M. P., Convergence of TINA algorithms,Acta Sci. Natur. Univ. Pekin., 1996, 32(5): 557.Google Scholar
  20. 20.
    Lei, G., Qian, M. P., Generalized time invariant noise algorithm and related bifurcation problem,Tech. Report (in Chinese), Beijing: Peking University Press, 1997.Google Scholar
  21. 21.
    Fang, H., Gong, G., Qian, M. P., An improved annealing method and its large time behavior,Stochastic Processes and Their Appl., 1997, 71(1): 55.CrossRefGoogle Scholar
  22. 22.
    Kolmogorov, A. N., On the representation of continuous functions of many variables by superposition of continuous functions of one variable and addition,Dokl. Akad. Nauk. (in Russian), 1957, 144: 953.Google Scholar
  23. 23.
    Hopfield, J. J., Neural networks and physical systems with emergent collective computational ability, inProc. of Nat. Acad. Sci., USA, 1982, 79: 2554.CrossRefGoogle Scholar
  24. 24.
    Hopfield, J. J., Pattern recognition computation using action potential timing for stimulus representation,Nature, 1996, 6 (July).Google Scholar
  25. 25.
    Zu, Z. B., Hu, G. Q., Kwong, C. P., Asymmetric Hopfield-type networks: theory and applications,Neural Networks, 1996, 9(3): 483.CrossRefGoogle Scholar
  26. 26.
    Hertz, J., Krogh, A., Palmer, G. R., Introduction to the theory of neural computation, LN Vol. 1, Santa Fe Institute,Studies in the Science of Complexity, Redwood City, California: Addison-Wesley Pub. Co., 1991.Google Scholar
  27. 27.
    Azencott, R., Boltzmann machines: high-order interactions and synchronous learning,Stochastic Models, Statistical Methods and Algorithms in Image Analysis. Lecture Notes in Statistics (ed. Barone, P.), Vol. 74, Berlin: Springer, 1992.Google Scholar
  28. 28.
    Zheng, J. L.,Artificial Neural Network (in Chinese), Beijing: Higher Education Publishing House, 1992.Google Scholar
  29. 29.
    Amit, D. J.,Modeling Brain Function, the World of Attractor Neural Networks, Cambridge: Cambridge University Press, 1989.Google Scholar
  30. 30.
    Geman, S., Bienenstock, E., Doursat, R., Neural networks and the bias/variance dilemma,Neural Computation, 1992, 4: 1CrossRefGoogle Scholar
  31. 31.
    Linsker, R., Self-organization in a perceptual network,Conputer, 1988, 21(3): 105.Google Scholar
  32. 32.
    Feng, J., Pan, H., Analysis of Linsker-Type Hebbian learning: rigorous results, 1993IEEE International Conference on Neural Networks, San Francisco, California, 1993.Google Scholar
  33. 33.
    Albeverio, S., Feng, J., Qian, M. P., Role of noise in neural networks,Physics Rev. E, 1995, 52(6): 6593.CrossRefGoogle Scholar
  34. 34.
    Holland, J. H.,Adaptation in Natural and Artificial Systems, Ann Arbor, Chicargo: The Univ. of Michigan Press, 1975.Google Scholar
  35. 35.
    Kohonen, T.,Self-organization and Associative Memory, 3rd ed., Berlin: Springer-Verlag, 1989.Google Scholar
  36. 36.
    Burton, R. M., Pages, G., Self-organization and u.s. convergence of the one-dimensional Kohonen algorithm with nonuniformly distributed Stimuli,Stochastic Processes and Their Appl., 1993, 47(2): 249.CrossRefGoogle Scholar
  37. 37.
    Burton, R. M., Faris, W. G., A self-organizing cluster process,Ann. of Appl Prob., 1996, 6(4): 1232.CrossRefGoogle Scholar
  38. 38.
    Grossberg, S., Competitive learning: from interactive activation to adaptive resonance cognitive,Science, 1987, 1: 23.Google Scholar
  39. 39.
    Carpenter, G. A., Grossberg, S., The ART of adaptive pattern recognition by a selforganizing neural network,Trans. IEEE on Computer, 1988, 37(3): 77.Google Scholar
  40. 40.
    Qian, M. P., Competition learning approach of artificial neural networks,Fundamental Study of Intelligent Computer ’94 (eds. Li, W., Huai, J. P., Bai, S.) (in Chinese), Beijing: Press of Tsinghua University, 1994, 9–12.Google Scholar
  41. 41.
    Qian, M. P., Wu, D., The statistics and discussion on various distance of images and applications to fuzzifying technique, inProc. of the Asian Conference on Statistical Computing, Beijing, 1993, 181–184.Google Scholar
  42. 42.
    Oja, E., Neural betworks, principle components, and subspace,International Journal of Neural Systems, 1989, 1: 61.CrossRefGoogle Scholar
  43. 43.
    Oja, E., Karrunen, J., On stochastic approximation of the eigenvectors and eigenvalues of the expectation of a random matrix,Journal of Mathematical Analysis and Applications, 1985, 100: 69.CrossRefGoogle Scholar
  44. 44.
    Benveniste, A., Metivier, M., Priouret, P.,Adaptive Algorithm and Stochastic Approximations, Berlin: Springer, 1990.Google Scholar
  45. 45.
    Chen, H. F., Zhu, Y. M.,Stochastic Approximations (in Chinese), Shanghai: Shanghai Science and Technology Press, 1996.Google Scholar
  46. 46.
    Kunsch, H., Geman, S., Kehagias, A., Hidden Markov random fields,Ann. Appl. Probab., 1993, 3(3): 577.Google Scholar
  47. 47.
    Rabiner, L. R., A tutorial on hidden Markov models and selected applications in speech recognition, inProc. IEEE, 1989, 77(2): 267.CrossRefGoogle Scholar
  48. 48.
    Rabiner, L. R., Juang Biing-hwang,Fundamentals of Speech Recognition, Hong Kong: Prince Hall International Inc., 1993.Google Scholar
  49. 49.
    Huo, Q., Chan Chorkin, Contextual vector quantization for speech recognition with discrete hidden Markov model,International Symposium on Speech Image Processing and Neural Networks, 13–16, April, 1994, Hong Kong, 698–701.Google Scholar
  50. 50.
    Deng, M. H., Qian, M. P., Method of Recognition of handwriting Chinese characters and their realization based on hidden Markov fields,Symposium of the 3rd Session Intelligent Intersection of Computer in China and Intelligence Application Conference (eds. Wu, Q. Y., Qian, Y. L.), Beijing: Electronic Engineering Press, 1997, 204–208.Google Scholar
  51. 51.
    Churchill, G. A., Accurate restoration of DNA sequences,Case Study in Bayesian Statistics, Vol. II,Lecture Notes in Statistics (eds. Gatsonis, C., Hodges, J. S., Kass, R. E.et al.), 105, Berlin: Springer-Verlag, 1995, 90–148.Google Scholar
  52. 52.
    Leroux, B. G., Maximum-likelihood estimation for hidden Markov modeling,Stoc. Processes and their Appl., 1992, 40(1): 127.CrossRefGoogle Scholar
  53. 53.
    Elliott, R. J., Aggoun, L., Moore, J. B.,Hidden Markov Models, Berlin: Springer-Verlag, 1995.Google Scholar
  54. 54.
    Ho Yu-chi Larry, Soft Optimization of Hard Problem, inProc. of International Conference on Control and Information (invited talk), Hong Kong, 1996.Google Scholar

Copyright information

© Science in China Press 1999

Authors and Affiliations

  • Minping Qian
    • 1
  • Guanglu Gong
    • 2
  1. 1.Deportment of Probability and StatisticsPeking UniversityBeijingChina
  2. 2.Department of Applied MathematicsTsinghua UniversityBeijingChina

Personalised recommendations