Skip to main content
SpringerLink
Log in

GAME – Hybrid Self-Organizing Modeling System Based on GMDH

  • Chapter
Hybrid Self-Organizing Modeling Systems

Part of the book series: Studies in Computational Intelligence ((SCI,volume 211))

Abstract

In this chapter, an algorithm to construct hybrid self-organizing neural network is proposed. It combines niching evolutionary strategies, nature inspired and gradient based optimization algorithms (Quasi-Newton, Conjugate Gradient, GA, PSO, ACO, etc.) to evolve neural network with optimal topology adapted to a data set. The GAME algorithm is something in between the GMDH algorithm and the NEAT algorithm. It is capable to handle irrelevant inputs, short and noisy data samples, but also complex data such as “two intertwined spirals” problem. The selforganization of the topology allows it to produce accurate models for various tasks (classification, prediction, regression, etc.). Bencharking with machine learning algorithms implemented in the Weka software showed that the accuracy of GAME models was superior for both regression and classification problems. The most successful configuration of the GAME algorithm is not changing with problem character, natural evolution selects all important parameters of the algorithm. This is a significant step towards the automated data mining.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. The sumatra tt data preprocessing tool (September 2006), http://krizik.felk.cvut.cz/sumatra/

  2. Uci machine learning repository (September 2006), http://www.ics.uci.edu/~mlearn/MLSummary.html

  3. Weka open source data mining software (September 2006), http://www.cs.waikato.ac.nz/ml/weka/

  4. The yale open source learning environment (September 2006), http://www-ai.cs.uni-dortmund.de/SOFTWARE/YALE/intro.html

  5. The fake game environment for the automatic knowledge extraction (November 2008), http://www.sourceforge.net/projects/fakegame

  6. Adeney, K., Korenberg, M.: An easily calculated bound on condition for orthogonal algorithms. In: IEEE-INNS-ENNS International Joint Conference on Neural Networks (IJCNN 2000), vol. 3, p. 3620 (2000)

    Google Scholar 

  7. Bilchev, G., Parmee, I.C.: The ant colony metaphor for searching continuous design spaces. In: Selected Papers from AISB Workshop on Evolutionary Computing, pp. 25–39. Springer, London (1995)

    Google Scholar 

  8. Blum, C., Socha, K.: Training feed-forward neural networks with ant colony optimization: An application to pattern classification. In: Proceedings of Hybrid Intelligent Systems Conference, HIS 2005, pp. 233–238. IEEE Computer Society, Los Alamitos (2005)

    Google Scholar 

  9. Brown, G.: Diversity in Neural Network Ensembles. PhD thesis, The University of Birmingham, School of Computer Science, Birmingham B15 2TT, United Kingdom (January 2004)

    Google Scholar 

  10. Fahlman, S.E., Lebiere, C.: The cascade-correlation learning architecture. Technical Report CMU-CS-90-100, Carnegie Mellon University Pittsburgh, USA (1991)

    Google Scholar 

  11. Fayyad, U., Shapiro, G., Smyth, P.: From data mining to knowledge discovery in databases. AI Magazine 17(3), 37–54 (1996)

    Google Scholar 

  12. Granitto, P., Verdes, P., Ceccatto, H.: Neural network ensembles: evaluation of aggregation algorithms. Artificial Intelligence 163, 139–162 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  13. Hansen, L., Salamon, P.: Neural network ensembles. IEEE Trans. Pattern Anal. Machine Intelligence 12(10), 993–1001 (1990)

    Article  Google Scholar 

  14. Ho, Y.-C., Pepyne, D.: Simple explanation of the no free lunch theorem of optimization. In: Proceedings of the 40th IEEE Conference on Decision and Control, December 4-7, vol. 5, pp. 4409–4414 (2001)

    Google Scholar 

  15. Holland, J.: Adaptation in Neural and Artificial Systems. University of Michigan Press (1975)

    Google Scholar 

  16. Hrstka, O., Kučerová, A.: Improvements of real coded genetic algorithms based on differential operators preventing premature convergence. Advances in Engineering Software 35(3-4), 237–246 (2004)

    Article  Google Scholar 

  17. Islam, M., Yao, X., Murase, K.: A constructive algorithm for training cooperative neural network ensembles. IEEE Transitions on Neural Networks 14(4) (July 2003)

    Google Scholar 

  18. Ivakhnenko, A.G.: Polynomial theory of complex systems. IEEE Transactions on Systems, Man, and Cybernetics SMC-1(1), 364–378 (1971)

    Article  MathSciNet  Google Scholar 

  19. Juang, C.-F., Liou, Y.-C.: On the hybrid of genetic algorithm and particle swarm optimization for evolving recurrent neural network. In: Proceedings of the IEEE International Joint Conference on Neural Networks, Dept. of Electr. Eng., Nat. Chung-Hsing Univ., Taichung, Taiwan, July 25-29, vol. 3, pp. 2285–2289 (2004)

    Google Scholar 

  20. Juille, H., Pollack, J.B.: Co-evolving intertwined spirals. In: Lawrence, P.J.A., Fogel, J., Baeck, T. (eds.) Proceedings of the Fifth Annual Conference on Evolutionary Programming. Evolutionary Programming V, pp. 461–467. MIT Press, Cambridge (1996)

    Google Scholar 

  21. Kohavi, R.: A study of cross-validation and bootstrap for accuracy estimation and model selection. In: Proceedings of International Joint Conference on Artificial Intelligence (1995)

    Google Scholar 

  22. Kohonen, T.: Self-Organizing Maps. Springer, Heidelberg (2001)

    MATH  Google Scholar 

  23. Kong, M., Tian, P.: A direct application of ant colony optimization to function optimization problem in continuous domain. In: Dorigo, M., Gambardella, L.M., Birattari, M., Martinoli, A., Poli, R., Stützle, T. (eds.) ANTS 2006. LNCS, vol. 4150, pp. 324–331. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  24. Kordík, P.: Game - group of adaptive models evolution. Technical Report DCSE-DTP-2005-07, Czech Technical University in Prague, FEE, CTU Prague, Czech Republic (2005)

    Google Scholar 

  25. Kordík, P.: Fully Automated Knowledge Extraction using Group of Adaptive Models Evolution. PhD thesis, Czech Technical University in Prague, FEE, Dep. of Comp. Sci. and Computers, FEE, CTU Prague, Czech Republic (September 2006)

    Google Scholar 

  26. Kordík, P., Křemen, V., Lhotská, L.: The game algorithm applied to complex fractionated atrial electrograms data set. In: Koutník, J., Kůrková, V., Neruda, R. (eds.) ICANN 2008, Part II. LNCS, vol. 5164, pp. 859–868. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  27. Kordík, P., Náplava, P., Šnorek, M., Genyk-Berezovskyj, M.: The Modified GMDH Method Applied to Model Complex Systems. In: International Conference on Inductive Modeling - ICIM 2002, Lviv, pp. 150–155. State Scientific and Research Institute of Information Infrastructure (2002)

    Google Scholar 

  28. Kuhn, L.: Ant Colony Optimization for Continuous Spaces. PhD thesis, The Department of Information Technology and Electrical Engineering The University of Queensland (October 2002)

    Google Scholar 

  29. Li, Y.-J., Wu, T.-J.: An adaptive ant colony system algorithm for continuous-space optimization problems. J. Zhejiang Univ. Sci. 4(1), 40–46 (2003)

    Article  Google Scholar 

  30. Mahfoud, S.W.: A comparison of parallel and sequential niching methods. In: Sixth International Conference on Genetic Algorithms, pp. 136–143 (1995)

    Google Scholar 

  31. Mahfoud, S.W.: Niching methods for genetic algorithms. Technical Report 95001, Illinois Genetic Algorithms Laboratory (IlliGaL), University of Ilinios at Urbana-Champaign (May 1995)

    Google Scholar 

  32. Mandischer, M.: A comparison of evolution strategies and backpropagation for neural network training. Neurocomputing (42), 87–117 (2002)

    Google Scholar 

  33. Monmarché, N., Venturini, G., Slimane, M.: On how pachycondyla apicalis ants suggest a new search algorithm. Future Gener. Comput. Syst. 16(9), 937–946 (2000)

    Article  Google Scholar 

  34. Muller, J.A., Lemke, F.: Self-Organising Data Mining, Berlin (2000) ISBN 3-89811-861-4

    Google Scholar 

  35. Nariman-Zadeh, N., Darvizeh, A., Jamali, A., Moeini, A.: Evolutionary design of generalized polynomial neural networks for modelling and prediction of explosive forming process. Journal of Materials Processing Technology (165), 1561–1571 (2005)

    Google Scholar 

  36. Oh, S.-K., Pedrycz, W., Park, B.-J.: Polynomial neural networks architecture: analysis and design. Computers and Electrical Engineering 29(29), 703–725 (2003)

    Article  Google Scholar 

  37. Pyle, D.: Data Preparation for Data Mining. Morgan Kaufman, Fondi di Ricerca Salvatore Ruggieri - Numero 421 d’inventario (1999)

    Google Scholar 

  38. Salane, Tewarson: A unified derivation of symmetric quasi-newton update formulas. Applied Math. 25, 29–36 (1980)

    MATH  MathSciNet  Google Scholar 

  39. Schnabel, R., Koontz, J., Weiss, B.: A modular system of algorithms for unconstrained minimization. Technical Report CU-CS-240-82, Comp. Sci. Dept., University of Colorado at Boulder (1982)

    Google Scholar 

  40. Seiffert, U., Michaelis, B.: Adaptive three-dimensional self-organizing map with a two-dimensional input layer. In: Australian and New Zealand Conference on Intelligent Information Systems, November 18-20, pp. 258–263 (1996)

    Google Scholar 

  41. Sexton, R.S., Gupta, J.: Comparative evaluation of genetic algorithm and backpropagation for training neural networks. Information Sciences (129), 45–59 (2000)

    Google Scholar 

  42. Shewchuk, J.R.: An introduction to the conjugate gradient method without the agonizing pain. Technical report, School of Computer Science Carnegie Mellon University, Pittsburgh, PA 15213 (August. 1994)

    Google Scholar 

  43. Siegl, T., Kordík, P., Šnorek, M., Calda, P.: Fetal weight prediction models: Standard techniques or computational intelligence methods? In: Koutník, J., Kůrková, V., Neruda, R. (eds.) ICANN 2008, Part I. LNCS, vol. 5163, pp. 462–471. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  44. Stanley, K., Bryant, B., Miikkulainen, R.: Real-time neuroevolution in the nero video game. IEEE Transactions on Evolutionary Computation 9(6), 653–668 (2005)

    Article  Google Scholar 

  45. Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evolutionary Computation 10(2), 99–127 (2002)

    Article  Google Scholar 

  46. Statsoft. Statistica neural networks software (September 2006), http://www.statsoft.com/products/stat_nn.html

  47. Storn, R., Price, K.: Differential evolution - a simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization 11, 341–359 (1997)

    Article  MATH  MathSciNet  Google Scholar 

  48. Tsutsui, S., Pelikan, M., Ghosh, A.: Performance of aggregation pheromone system on unimodal and multimodal problems. In: The IEEE Congress on Evolutionary Computation, 2005 (CEC 2005), September 2-5, vol. 1, pp. 880–887. IEEE, Los Alamitos (2005)

    Chapter  Google Scholar 

  49. Tzeng, F.-Y., Ma, K.-L.: Opening the black box - data driven visualization of neural networks. In: Proceedings of IEEE Visualization 2005 Conference, Minneapolis, USA, pp. 23–28 (October 2005)

    Google Scholar 

  50. Vesterstrom, J., Thomsen, R.: A comparative study of differential evolution, particle swarm optimization, and evolutionary algorithms on numerical benchmark problems. In: Proceedings of the 2004 Congress on Evolutionary Computation, vol. 2, pp. 1980–1987 (2004)

    Google Scholar 

  51. Wade, J.G.: Convergence properties of the conjugate gradient method (September 2006), www-math.bgsu.edu/~gwade/tex_examples/example2.txt

  52. Wickera, D., Rizkib, M.M., Tamburinoa, L.A.: E-net: evolutionary neural network synthesis. Neurocomputing 42, 171–196 (2002)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, FEE, Czech Technical University, Prague, Czech Republic

    Pavel Kordík

Authors
  1. Pavel Kordík
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Knowledge Management & Mining Inc., Richmond Hill, Ontario, Canada

    Godfrey C. Onwubolu

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Kordík, P. (2009). GAME – Hybrid Self-Organizing Modeling System Based on GMDH. In: Onwubolu, G.C. (eds) Hybrid Self-Organizing Modeling Systems. Studies in Computational Intelligence, vol 211. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-01530-4_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-01530-4_6

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-01529-8

  • Online ISBN: 978-3-642-01530-4

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation