Information Theory of Complex Networks: On Evolution and Architectural Constraints

  • Ricard V. Solé
  • Sergi Valverde
Part I Network Structure
Part of the Lecture Notes in Physics book series (LNP, volume 650)


Complex networks are characterized by highly heterogeneous distributions of links, often pervading the presence of key properties such as robustness under node removal. Several correlation measures have been defined in order to characterize the structure of these nets. Here we show that mutual information, noise and joint entropies can be properly defined on a static graph. These measures are computed for a number of real networks and analytically estimated for some simple standard models. It is shown that real networks are clustered in a well-defined domain of the entropy-noise space. By using simulated annealing optimization, it is shown that optimally heterogeneous nets actually cluster around the same narrow domain, suggesting that strong constraints actually operate on the possible universe of complex networks. The evolutionary implications are discussed.


Mutual Information Complex Network Metabolic Network Degree Distribution Electronic Circuit 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1. R. Albert and A.-L. Barabási. Statistical Mechanics of Complex Networks. Rev. Mod. Phys. 74, 47-97 (2002a).Google Scholar
  2. 2. S. N. Dorogovtsev andJ. F. F. Mendes. Evolution of networks. Adv. Phys. 51, 1079-1187 (2002).Google Scholar
  3. 3. S. N. Dorogovtsev andJ. F. F. Mendes. Evolution of Networks: from biological nets to the Internet and WWW Oxford U. Press, Oxford (2003).Google Scholar
  4. 4. S. Bornholdt and H. G. Schuster, eds. Handbook of Graphs and Networks: From the Genome to the Internet. Springer, Berlin (2002).Google Scholar
  5. 5. L. A. N. Amaral, A. Scala, M. Barthélemy and H. E. Stanley. Classes of behavior of small-world networks. Proc. Nat. Acad. Sci. USA 97, 11149-11152 (2000).Google Scholar
  6. 6. R. Ferrer, C. Janssen and R. V. Solé. Topology of Technology Graphs: Small World Patterns in Electronic Circuits Physical Review E 64, 32767 (2001).Google Scholar
  7. 7. A.-L. Barabási and R. Albert. Emergence of scaling in random networks. Science 286, 509-512 (1999).Google Scholar
  8. 8. S. N. Dorogovtsev andJ. F. F. Mendes. Accelerated growth of networks, in: Handbook of Graphs and Networks: From the Genome to the Internet, eds. S. Bornholdt and H.G. Schuster. pp. 320-343 Wiley-VCH, Berlin (2002).Google Scholar
  9. 9. R. Ferrer and R. V. Solé. Optimization in Complex Networks, Lect. Notes Phys. 625, 114–125 (2003).Google Scholar
  10. 10. S. Valverde, R. Ferrer and R. V. Solé. Scale free networks from optimal design Europhys. Lett. 60, 512-517 (2002).Google Scholar
  11. 11. R. V. Solé, R. Pastor-Satorras, R., Smith, E.D. and Kepler, T. A model of large-scale proteome evolution. Adv. Complex Systems 5, 43-54 (2002).Google Scholar
  12. 12. A. Vazquez, A. Flammini, A. Maritan and A. Vespignani. Modeling of protein interaction networks. Complexus, 1, 38-44 (2002).Google Scholar
  13. 13. G. Caldarelli, A. Capocci, P. De Los Rios and M. A. Muñoz. Scale-free networks from varying vertex intrinsic fitness. Phys. Rev. Lett. 89, 258702 (2002).Google Scholar
  14. 14. F. Menczer. Growing and navigating the small world web by local content. Proc. Nat. Acad. Sci. USA 99, 14014-14019 (2002).Google Scholar
  15. 15. E. Ravasz, A. L. Somera, D. A. Mongru, Z. N. Oltvai and A.-L. Barabási. Hierarchical Organization of Modularity in Metabolic Networks. Science 297, 1551-1555 (2002).Google Scholar
  16. 16. M. E. J. Newman. Assortative mixing in networks. Phys. Rev. Lett. 89, 208701 (2002).Google Scholar
  17. 17. E. H. Stanley, S. V. Buldyrev, A. L. Goldberger, Z. D. Goldberger, S. Havlin, R. N. Mantegna, S. M. Ossadnik, C. K. Peng and M. Simon, Statistical mechanics in biology: how ubiquitous are long-range correlations? Physica A205, 214-253 (1996).Google Scholar
  18. 18. H. D. Abarbanel, R. Brown, J. L. Sidorowich and L. S. Tsimring. The analysis of observed chaotic data in physical systems. Rev. Mod. Phys. 65, 1331-1392 (1993).Google Scholar
  19. 19. R. B. Ash. Information Theory, Dover, London (1965).Google Scholar
  20. 20. C. Adami. Introduction to Artificial Life. Springer, New York (1998).Google Scholar
  21. 21. W. Li. Mutual information versus correlation functions. J. Stat. Phys. 60, 823-837 (1990).Google Scholar
  22. 22. W. Li. On the relationship between complexity and entropy for Markov chains and regular languages. Complex Syst. 5, 381-399 (1991).Google Scholar
  23. 23. A. Fraser and H. Swinney. Independent coordinates for strange attractors from mutual information. Phys. Rev. A33, 1134-1140 (1986).Google Scholar
  24. 24. C. Germain-Renaud and J. P. Sansonnet. Ordinateurs massivement paralleles, Armand Colin, Paris (1991).Google Scholar
  25. 25. V. M. Milutinovic. Computer Architecture, North Holland, Elsevier (1988).Google Scholar
  26. 26. W. D. Hillis. The Connection Machine, MIT Press (Cambridge, MA, 1985).Google Scholar
  27. 27. S. Valverde, R. Ferrer and R. V. Solé, Scale-free networks from optimal design. Europhys. Lett. 60, 512-517 (2002).Google Scholar
  28. 28. S. A. Kauffman. Origins of Order. Oxford U. Press, New York (1993).Google Scholar
  29. 29. R. M. May. Stability and complexity in model ecosystems. Princeton U. Press, New York (1973).Google Scholar
  30. 30. S. Amari. Characteristics of random nets of analog neuron-like elements. IEEE Trans. Man and Cybernetics 2, 643-657 (1972).Google Scholar
  31. 31. J. M. Montoya and R. V. Solé. Topological properties of food webs: from real data to community assembly models Oikos 102, 614-622 (2003).Google Scholar
  32. 32. R. Ferrer and R. V. Solé, Optimization in Complex Networks, Lect. Notes Phys. 625, 114–125 (2003).Google Scholar
  33. 33. H. Jeong, S. Mason, A.-L. Barabási and Z. N. Oltvai. Lethality and centrality in protein networks Nature 411, 41 (2001).Google Scholar
  34. 34. S. Valverde and R. V. Solé. Hierarchical small worlds in sotfware architecture. Santa Fe Institute Working Paper 03-07-044.Google Scholar
  35. 35. C. R. Myers. Software systems as complex networks: structure, function, and evolvability of software collaboration graphs, Phys. Rev. E 68, 046116 (2003).Google Scholar
  36. 36. F. Schweitzer, W. Ebeling, H. Rose and O. Weiss. Network Optimization Using Evolutionary Strategies, in: Parallel Problem Solving from Nature - PPSN IV, (Eds. H.-M. Voigt, W. Ebeling, I. Rechenberg, H.-P. Schwefel), Lecture Notes in Computer Science, vol. 1141, Springer, Berlin (1996) pp. 940-949.Google Scholar
  37. 37. F. Schweitzer. Brownian Agents and Active Particles. Springer, Berlin (2002).Google Scholar
  38. 38. R. V. Solé, R. Ferrer-Cancho, J. M. Montoya and S. Valverde. Selection, tinkering and emergence in complex networks. Complexity 8(1), 20-33 (2002).Google Scholar
  39. 39. F. Jacob. Evolution as tinkering. Science 196, 1161-1166 (1976).Google Scholar
  40. 40. P. Alberch. The logic of monsters: evidence for internal constraint in development and evolution. Geobios 19, 21-57 (1989).Google Scholar
  41. 41. B. C. Goodwin. How the Leopard Changed Its Spots: the Evolution of Complexity. Charles Scribner’s Sons, New York (1994).Google Scholar
  42. 42. S. J. Gould. The structure of evolutionary theory. Belknap, Harvard (2003).Google Scholar

Authors and Affiliations

  • Ricard V. Solé
    • 1
  • Sergi Valverde
    • 1
  1. 1.Complex Systems Lab-ICREA, Universitat Pompeu Fabra (GRIB), Dr Aiguader 80, 08003 BarcelonaSpain

Personalised recommendations