Skip to main content
SpringerLink
Log in
Book cover

International Joint Conference on Computational Intelligence

IJCCI 2015: Computational Intelligence pp 337–358Cite as

High Capacity Content Addressable Memory with Mixed Order Hyper Networks

  • Conference paper
  • First Online:

  • 731 Accesses

Part of the book series: Studies in Computational Intelligence ((SCI,volume 669))

Abstract

A mixed order hyper network (MOHN) is a neural network in which weights can connect any number of neurons, rather than the usual two. MOHNs can be used as content addressable memories (CAMs) with higher capacity than standard Hopfield networks. They can also be used for regression learning of functions in \(f:\{-1,1\}^n \rightarrow \mathbb {R}\) in which the turning points are equivalent to memories in a CAM. This paper presents a number of methods for learning an energy function from data that can act as either a CAM or a regression model and presents the advantages of using such an approach.

This is a preview of subscription content, log in via an institution.

Notes

  1. 1.

    The regression Eq. 9 is actually the negative of the energy function, which is minimised by applying the settling Algorithm 2.

References

  1. Beauchamp, K.: Applications of Walsh and Related Functions. Academic Press, London (1984)

    Google Scholar 

  2. Caparrós, G.J., Ruiz, M.A.A., Hernández, F.S.: Hopfield neural networks for optimization: study of the different dynamics. Neurocomputing 43(1–4), 219–237 (2002)

    Google Scholar 

  3. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer, New York (2009)

    Book  MATH  Google Scholar 

  4. Heckendorn, R.B., Wright, A.H.: Efficient linkage discovery by limited probing. Evol. Comput. 12(4), 517–545 (2004)

    Article  MATH  Google Scholar 

  5. Hopfield, J.J., Tank, D.W.: Neural computation of decisions in optimization problems. Biol. Cybern. 52, 141–152 (1985)

    Article  MATH  Google Scholar 

  6. Hopfield, J.J.: Neural networks and physical systems with emergent collective computational abilities. Proc. Nat. Acad. Sci. USA 79(8), 2554–2558 (1982)

    Article  MathSciNet  MATH  Google Scholar 

  7. Kubota, T.: A higher order associative memory with Mcculloch-Pitts neurons and plastic synapses. In: 2007 International Joint Conference on Neural Networks, IJCNN 2007, pp. 1982–1989, August 2007

    Google Scholar 

  8. Pelikan, M., Goldberg, D.E., Cantú-paz, E.E.: Linkage problem, distribution estimation, and Bayesian networks. Evol. Comput. 8(3), 311–340 (2000)

    Article  Google Scholar 

  9. Storkey, A.J., Valabregue, R.: The basins of attraction of a new hopfield learning rule. Neural Netw. 12(6), 869–876 (1999)

    Article  Google Scholar 

  10. Swingler, K.: Applying Neural Networks: A Practical Guide. Academic Press, London (1996)

    Google Scholar 

  11. Swingler, K.: On the capacity of Hopfield neural networks as EDAs for solving combinatorial optimisation problems. In: Proceedings of IJCCI (ECTA), pp. 152–157. SciTePress (2012)

    Google Scholar 

  12. Swingler, K.: A comparison of learning rules for mixed order hyper networks. In: Proceedings of IJCCI (NCTA) (2015)

    Google Scholar 

  13. Swingler, K.: Local optima suppression search in mixed order hyper networks. In: 2015 15th UK Workshop on Computational Intelligence (UKCI) (2015)

    Google Scholar 

  14. Swingler, K.: Opening the black box: analysing MLP functionality using walsh functions. In: Merelo, J.J., Rosa, A., Cadenas, J.M., Dourado, A., Madani, K., Filipe, J. (eds.) Computational Intelligence. Studies in Computational Intelligence, vol. 620, pp. 303–323. Springer, Cham (2016)

    Chapter  Google Scholar 

  15. Swingler, K.: Structure discovery in mixed order hyper networks. Big Data Analytics 1(1), 8 (2016)

    Article  Google Scholar 

  16. Swingler, K., Smith, L.: Training and making calculations with mixed order hyper-networks. Neurocomputing 141, 65–75 (2014)

    Article  Google Scholar 

  17. Swingler, K., Smith, L.: An analysis of the local optima storage capacity of hopfield network based fitness function models. In: Nguyen, N.T., Kowalczyk, R., Fred, A., Joaquim, F. (eds.) Transactions on Computational Collective Intelligence XVII. LNCS, vol. 8790, pp. 248–271. Springer, Heidelberg (2014). doi:10.1007/978-3-662-44994-3_13

    Chapter  Google Scholar 

  18. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. Roy. Stat. Soc. Series B (Methodological), 267–288 (1996)

    Google Scholar 

  19. Venkatesh, S.S., Baldi, P.: Programmed interactions in higher-order neural networks: maximal capacity. J. Complex. 7(3), 316–337 (1991)

    Article  MathSciNet  MATH  Google Scholar 

  20. Walsh, J.: A closed set of normal orthogonal functions. Amer. J. Math. 45, 5–24 (1923)

    Article  MathSciNet  MATH  Google Scholar 

  21. Wilson, G.V., Pawley, G.S.: On the stability of the travelling salesman problem algorithm of hopfield and tank. Biol. Cybern. 58(1), 63–70 (1988)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computing Science and Maths, University of Stirling, Stirling, FK9 4LA, Scotland, UK

    Kevin Swingler

Authors
  1. Kevin Swingler
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kevin Swingler .

Editor information

Editors and Affiliations

  1. Computer Architecture and Computer Technology, Universidad de Granada, Granada, Spain

    Juan Julián Merelo

  2. aSEEB-ISR-IST, Technical University of Lisbon (IST), Lisbon, Portugal

    Agostinho Rosa

  3. Facultad de Informática, University of Murcia, Murcia, Spain

    José M. Cadenas

  4. Departamento de Engenharia Informatica, University of Coimbra, Coimbra, Portugal

    António Dourado Correia

  5. Images, Signals and Intelligence Systems Laboratory, University PARIS-EST Creteil (UPEC), Créteil, France

    Kurosh Madani

  6. Campus de Gambelas, Universidade do Algarve, Faro, Portugal

    António Ruano

  7. Escola Superior de Tecnologia de Setúbal, Polytechnic Institute of Setúbal/INSTICC, Setúbal, Portugal

    Joaquim Filipe

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Swingler, K. (2017). High Capacity Content Addressable Memory with Mixed Order Hyper Networks. In: Merelo, J.J., et al. Computational Intelligence. IJCCI 2015. Studies in Computational Intelligence, vol 669. Springer, Cham. https://doi.org/10.1007/978-3-319-48506-5_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-48506-5_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-48504-1

  • Online ISBN: 978-3-319-48506-5

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation