Skip to main content
SpringerLink
Log in

A Meta-Model Perspective and Attribute Grammar Approach to Facilitating the Development of Novel Neural Network Models

  • Chapter
Meta-Learning in Computational Intelligence

Part of the book series: Studies in Computational Intelligence ((SCI,volume 358))

  • 1185 Accesses

Abstract

There is a need for methods and tools that facilitate the systematic exploration of novel artificial neural network models. While significant progress has been made in developing concise artificial neural networks that implement basic models of neural activation, connectivity and plasticity, limited success has been attained in creating neural networks that integrate multiple diverse models to produce highly complex neural systems. From a problem-solving perspective, there is a need for effective methods for combining different neural-network-based learning systems in order to solve complex problems. Different models may be more appropriate for solving different subproblems, and robust, systematic methods for combining those models may lead to more powerful machine learning systems. From a neuroscience modelling perspective, there is a need for effective methods for integrating different models to produce more robust models of the brain. These needs may be met through the development of meta-model languages that represent diverse neural models and the interactions between different neural elements. A meta-model language based on attribute grammars, the Network Generating Attribute Grammar Encoding, is presented, and its capability for facilitating automated search of complex combinations of neural components from different models is discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Parks, R.W., Levine, D.S., Long, D.L.: Fundamentals of Neural Network Modeling: Neuropsychology and Cognitive Neuroscience. The MIT Press, Cambridge (1999)

    Google Scholar 

  2. http://grey.colorado.edu/emergent/index.php/Comparison_of_Neural_Network_Simulators

  3. Demuth, H., Beale, M., Hagan, M.: Neural Network Toolbox: User’s Guide, Version 6. The Mathworks, Inc. (2010)

    Google Scholar 

  4. Aisa, B., Mingus, B., O’Reilly, R.: The emergent neural modeling system. Neural Networks 21, 1045–1212 (2008)

    Article  Google Scholar 

  5. NeuroSolutions. v 6.0 by NeuroDimension, Inc., http://www.neurosolutions.com/products/ns/

  6. Fischer, I., Hennecke, F., Bannes, C., Zell, A.: JavaNNS Java Neural Network Simulator User Manual Version 1.1. University of Tubingen Wilhelm-Schickard-Institute for Computer Science. Department of Computer Architecture, University of Tubingen Wilhelm-Schickard-Institute for Computer Science (2001)

    Google Scholar 

  7. Simbrain, Y.J.: A visual framework for neural network analysis and education. In: Lorenz, S., Egelhaaf, M. (eds.) Interactive Educational Media for the Neural and Cognitive Sciences; Brains, Minds & Media 3, bmm1411(2008)

    Google Scholar 

  8. Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA data mining software: An update. SIGKDD Explorations 11 (1) (2009)

    Google Scholar 

  9. Beeman, D., Wang, Z., Edwards, M., Bhalla, U., Cornelis, H., Bower, J.: The GENESIS 3.0 Project: a universal graphical user interface and database for research, collaboration, and education in computational neuroscience. BMC Neuroscience 8(Suppl. 2), P4 (2007)

    Article  Google Scholar 

  10. Carnevale, N.T., Hines, M.L.: The NEURON Book. Cambridge University Press, Cambridge (2006)

    Book  Google Scholar 

  11. Dudani, N., Ray, S., George, S., Bhalla, U.S.: Multiscale modeling and interoperability in MOOSE. BMC Neuroscience 10(Suppl. 1), P54 (2010)

    Google Scholar 

  12. Kock, G., Serbedzija, N.B.: Object-oriented and functional concepts in artificial neural network modelling. In: Intl. Joint Conference on Neural Networks, vol. 1, pp. 923–927 (1993)

    Google Scholar 

  13. Hopp, H., Prechelt, L.: CuPit-2 - A Parallel Language for Neural Algorithms: Language Reference and Tutorial. Technical Report 4/97, Institut fur Programmstrukturen und Datenorganisation. Karlsruhe, Germany (1997)

    Google Scholar 

  14. Strey, A.: EpsiloNN - A Tool for the Abstract Specification and Parallel Simulation of Neural Networks. Systems Analysis - Modelling - Simulation (SAMS) 34(4) (1999)

    Google Scholar 

  15. Linden, A., Tietz, C.: Combining multiple neural network paradigms and applications using SESAME. In: Intl. Joint Conference on Neural Networks, vol. 2, pp. 528–533 (1992)

    Google Scholar 

  16. Rubtsov, D., Butakov, S.: Application of XML for neural network exchange. Computer Standards and Interfaces,24(4), p. 311-322 (2002)

    Google Scholar 

  17. Gorchetchnikov and INCF Multiscale Modeling Taskforce. NineML – a description language for spiking neuron network modeling: the user layer. BMC Neuroscience 11(Suppl. 1), P71 (2010)

    Google Scholar 

  18. Raikov and INCF Multiscale Modeling Taskforce. NineML – a description language for spiking neuron network modeling: the abstraction layer. BMC Neuroscience, 11(Suppl. 1) P66 (2010)

    Google Scholar 

  19. Data Mining Group. PMML version 4.0 (2009), http://www.dmg.org/pmml-v4-0.html

  20. Miller, G.F., Todd, P.M., Hegde, S.U.: Designing neural networks using genetic algorithms. In: Third Intl Conf. on Genetic Algorithms and Their Applications, pp. 379–384. Morgan Kaufmann, San Francisco (1989)

    Google Scholar 

  21. Polani, D., Uthmann, T.: Adaptation of Kohonen feature map topologies by genetic algorithms. In: Männer, R., Manderick, B. (eds.) Parallel Problem Solving from Nature 2, pp. 421–429. Elsevier, Amsterdam (1992)

    Google Scholar 

  22. Schaffer, J.D., Caruana, R.A., Eshelman, L.J.: Using genetic search to exploit the emergent behavior of neural networks. Physica D 42, 244–248 (1990)

    Article  Google Scholar 

  23. Jacob, C.: Genetic L-system programming. In: Davidor, Y., Männer, R., Schwefel, H.-P. (eds.) PPSN 1994. LNCS, vol. 866, pp. 334–343. Springer, Heidelberg (1994)

    Google Scholar 

  24. Kitano, H.: Designing neural networks using genetic algorithms with graph generation system. Complex Systems 4, 461–476 (1990)

    MATH  Google Scholar 

  25. Gruau, F.: Automatic definition of modular neural networks. Adaptive Behavior 3(2), 151–183 (1995)

    Article  Google Scholar 

  26. Bengio, S., Bengio, Y., Cloutier, J.: Use of genetic programming for the search of a new learning rule for neural networks. In: First Conference on Evolutionary Computation, pp. 324–327 (1994)

    Google Scholar 

  27. Tsoulosa, I., Gavrilis, D., Glavas, E.: Neural network construction and training using grammatical evolution. Neurocomputing 72, 269–277 (2008)

    Article  Google Scholar 

  28. Mouret, J., Doncieux, S.: MENNAG: a modular, regular and hierarchical encoding for neural-networks based on attribute grammars. Evolutionary Intelligence 1, 187–207 (2008)

    Article  MATH  Google Scholar 

  29. Hussain, T.S., Browse, R.A.: Network generating attribute grammar encoding. In: IEEE International Joint Conference on Neural Network, pp. 431–436 (1998)

    Google Scholar 

  30. Hussain, T.S., Browse, R.A.: Evolving neural networks using attribute grammars. In: 2000 IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks, pp. 37-42 (2000)

    Google Scholar 

  31. Hussain, T.S.: Attribute Grammar Encoding of the Structure and Behaviour of Artificial Neural Networks. Ph.D. Thesis, Queen’s University (2003)

    Google Scholar 

  32. Hussain, T.S.: Generic Neural Markup Language: Facilitating the Design of Theoretical Neural Network Models. In: Proceedings of the 2004 IEEE International Joint Conference on Neural Networks (IJCNN), pp. 235–242. Institute of Electrical and Electronics Engineers, Inc., Piscataway (2004)

    Chapter  Google Scholar 

  33. Knuth, D.E.: The semantics of context-free languages. Mathematical Systems Theory 2(2), 127–145 (1968)

    Article  MATH  MathSciNet  Google Scholar 

  34. Deransart, P., Lorho, B., Jourdan, M.: Attribute Grammars: Definitions, Systems and Bibliography. LNCS, vol. 323. Springer, Heidelberg (1988)

    MATH  Google Scholar 

  35. Vellacott, O.R.: A framework of hierarchy for neural theory. In: Second International Conference on Artificial Neural Networks, pp. 237–241 (1991)

    Google Scholar 

  36. Gleeson, P., Steuber, V., Silver, R.A.: NeuroConstruct: A Tool for Modeling Networks of Neurons in 3D Space. Neuron 54(2), 219–235 (2007)

    Article  Google Scholar 

  37. Mcgarry, K., Wermter, S., Macintyre, J.: Hybrid neural systems: from simple coupling to fully integrated neural networks. Neural Computing Surveys 2, 62–93 (1999)

    Google Scholar 

  38. Shavlik, J.W.: A Framework for Combining Symbolic and Neural Learning, Technical Report 1123, Computer Sciences Department, University of Wisconsin - Madison (November 1992)

    Google Scholar 

  39. Hussain, T.S.: Explicit learning relationships within neural systems. In: Workshop on Achieving Functional Integration of Diverse Neural Models, held at 2005 International Joint Conference on Neural Networks, p.16 (2005)

    Google Scholar 

  40. Iversen, A., Taylor, N.K., Brown, K.: Integrating Neural Network Strategies for Discrimination, Recognition and Clustering. In: Workshop on Achieving Functional Integration of Diverse Neural Models, held at 2005. Intl. Joint Conference on Neural Networks, p. 4 (2005)

    Google Scholar 

  41. de Kamps, M.: Large scale brain simulations are not a technical problem. In: Workshop on Achieving Functional Integration of Diverse Neural Models, held at 2005 International Joint Conference on Neural Networks, p. 2 (2005)

    Google Scholar 

  42. de Kamps, M., Baier, V., Drever, J., Dietz, M., Mosenlechner, L., van der Velde, F.: The state of MIIND. Neural Networks 21(8), 1164–1181 (2008)

    Article  Google Scholar 

  43. Smieja, F.J., Mühlenbein, H.: Reflective Modular Neural Network Systems. Technical Report: GMD #633, German National Research Centre for Computer Science (1992)

    Google Scholar 

  44. Jacobs, R.A., Jordan, M.I., Nowlan, S.J., Hinton, G.E.: Adaptive mixtures of local experts. Neural Computation 3, 79–87 (1991)

    Article  Google Scholar 

  45. Hecht-Nielsen, R.: Neurocomputing. Addison-Wesley, Reading (1990)

    Google Scholar 

  46. Oblinger, D.: Bootstrapped learning: Creating the electronic student that learns from natural instruction. Defense Advanced Research Projects Agency briefing (2006), http://www.darpa.mil/ipto/programs/bl/docs/AAAI_Briefing.pdf (August 19, 2010)

  47. Burstein, M., Brinn, M., Cox, M., Hussain, T., Laddaga, R., McDermott, D., McDonald, D., Tomlinson, R.: An architecture and language for the integrated learning of demonstrations. In: Workshop on Acquiring Planning Knowledge via Demonstration. held at the Twenty-Second National Conference on Artificial Intelligence (2007)

    Google Scholar 

  48. Burstein, M., Laddaga, R., McDonald, D., Cox, M., Benyo, B., Robertson, P., Hussain, T., Brinn, M., McDermott, D.: POIROT: integrated learning of web service procedures. In: Cohn, A. (ed.) Proceedings of the 23rd National Conference on Artificial intelligence, vol. 3, pp. 1274–1279. AAAI Press, Chicago (2008)

    Google Scholar 

  49. Koch, C., Segev, I.: Methods in Neuronal Modeling. From Ions to Networks, 2nd edn. MIT Press, Cambridge (1998)

    Google Scholar 

  50. Beeman, D.: Introduction to Realistic Neural Modeling. In: Bower, J.M., Beeman, D. (eds.) Special Issue on Realistic Neural Modeling - WAM-BAMM 2005 Tutorials; Brains, Minds 1, bmm218, (2005)

    Google Scholar 

  51. Jaeger, D.: Realistic single cell modeling – from experiment to simulation. In: Bower, J.M., Beeman, D. (eds.) Special Issue on Realistic Neural Modeling WAM-BAMM 2005 Tutorials; Brains, Minds 1, bmm222 (2005)

    Google Scholar 

  52. Carpenter, G.A., Grossberg, S.: ART 2: Self-organization of stable category recognition codes for analog input patterns. Applied Optics 26(23), 4919–4930 (1987)

    Article  Google Scholar 

  53. Murre, J.M.J., Phaf, R.H., Wolters, G.: CALM: Categorizing and Learning Module. Neural Networks 5, 55–82 (1992)

    Article  Google Scholar 

  54. Mendao, M.: Hormonally moderated neural control. In: Hudlicka, E., Canamero, L. (eds.) Architectures for Modeling Emotion: Cross Disciplinary Foundations: Papers from the 2004 Spring Symposium, pp. 92–95. American Assoc. Artificial Intelligence, Menlo Park (2004)

    Google Scholar 

  55. Liu, B., Ding, Y., Wang, J.: Intelligent network control system inspired from neuroendocrine-immune system. In: Chen, Y., Zhang, D., Deng, H., Xiao, Y. (eds.) Proceedings of the 6th International Conference on Fuzzy Systems and Knowledge Discovery, vol. 7, pp. 136–140. IEEE Press, Piscataway (2009)

    Chapter  Google Scholar 

  56. Århem, P., Klement, G., Blomberg, C.: Channel density regulation of firing patterns in a cortical neuron model. Biophysical Journal 90(12), 4392–4404 (2006)

    Article  Google Scholar 

  57. Mouret, J., Doncieux, S., Girard, B.: Importing the computational neuroscience toolbox into neuro-evolution-application to basal ganglia. In: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation, pp. 587–594. ACM Press, New York (2010)

    Chapter  Google Scholar 

  58. Floreano, D., Epars, Y., Zufferey, J., Mattiussi, C.: Evolution of spiking neural circuits in autonomous mobile robots. Intl. Journal of Intelligent Systems 21(9), 1005–1024 (2006)

    Article  Google Scholar 

  59. Hebb, D.O.: The Organization of Behavior: A Neuropsychological Theory. Wiley, New York (1949)

    Google Scholar 

  60. Grillner, S.: The motor infrastructure: from ion channels to neuronal networks. Nature Reviews Neuroscience 4, 573–586 (2003)

    Article  Google Scholar 

  61. Song, S., Miller, K.D., Abbott, L.F.: Competitive Hebbian learning through spike-timing-dependent synaptic plasticity. Nature Neuroscience 3(9), 919–926 (2000)

    Article  Google Scholar 

  62. Kempter, R., Gerstner, W., van Hemmen, J.L.: Hebbian learning and spiking neurons. Physical Review E 59(4), 4498–4514 (1999)

    Article  MathSciNet  Google Scholar 

  63. Bray, T., Paoli, J., Sperberg-McQueen, C.M., Maler, E., Yergeau, F. (eds.): Extensible Markup Language (XML) 1.0 (5th edn.) (August 30, 2010), http://www.w3.org/TR/REC-xml/

Download references

Author information

Authors and Affiliations

  1. Raytheon BBN Technologies, Cambridge, MA, USA

    Talib S. Hussain

Authors
  1. Talib S. Hussain
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Informatics, Nicolaus Copernicus University, ul. Grudzia̧dzka 5, 87-100, Toruń, Poland

    Norbert Jankowski , Włodzisław Duch  & Krzysztof Gra̧bczewski ,  & 

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Hussain, T.S. (2011). A Meta-Model Perspective and Attribute Grammar Approach to Facilitating the Development of Novel Neural Network Models. In: Jankowski, N., Duch, W., Gra̧bczewski, K. (eds) Meta-Learning in Computational Intelligence. Studies in Computational Intelligence, vol 358. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-20980-2_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-20980-2_8

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-20979-6

  • Online ISBN: 978-3-642-20980-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation