Verification of Models in Agent Based Computational Economics — Lessons from Software Engineering

  • Bogumił Kamiński
  • Przemysław Szufel
Conference paper
Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 158)


Agent based models are highly complex and usually are being implemented using programming languages. This situation calls for adequate methods allowing for their verification that are not used in standard economic research. In order to organize this process we propose to logically decompose agent based model into three layers: conceptual model, computerized model and metamodel. The main possible problems identified using this decomposition are: (a) incomplete specification of conceptual model, (b) unexpected behavior of computerized model and (c) problems with reproduction simulation results. In order to address these issues based on literature review we draw recommendations concerning model documentation, testing and simulation reproduction that are crucial to improve their quality and precision of communication.


agent based modeling computational economics model verification software engineering 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ackoff, R.L.: Towards a system of system concepts. Management Science 17(11), 661–671 (1971)CrossRefGoogle Scholar
  2. 2.
    Bennett, B.S.: Simulation Fundamentals. Prentice Hall, London (1995)Google Scholar
  3. 3.
    Bonabeau, E.: Agent-based modeling: methods and techniques for simulating human systems, in: Proceedings of National Academy of Sciences 99(3): 7280–7287, (2001)Google Scholar
  4. 4.
    Bragen, M., Altaweel, M.: Repast Parameters Sweeps Getting started, (2013), (accessed on May 10, 2013)
  5. 5.
    Bruegge, B., Dutoit, A.H.: Object-Oriented Software Engineering: Using UML, Patterns and Java. Prentice Hall, Upper Saddle River (2009)Google Scholar
  6. 6.
    Casti, J.: Would-be worlds: how simulation is changing the world of science. Wiley, New York (1997)Google Scholar
  7. 7.
    Cirillo, R., Thimmapuram, P., Veselka, T., Koritarov, V., Conzelmann, G., Macal, C., Boyd, G., North, M., Overbye, T., Cheng, X.: Evaluating the Potential Impact of Transmission Constraints on the Operation of a Competitive Electricity Market in Illinois, Argonne National Laboratory, Argonne, IL, ANL-06/16 (report prepared for the Illinois Commerce Commission) (2006)Google Scholar
  8. 8.
    Cont, R.: Volatility Clustering in Financial Markets: Empirical Facts and Agent-Based Models. In: Teyssière, G., Kirman, A. (eds.) Long Memory in Economics. Springer (2007)Google Scholar
  9. 9.
    Daum, T., Sargent, R.G.: Experimental frames in a modern modeling and simulation system. IIE Transactions 33, 181–192 (2001)Google Scholar
  10. 10.
    Dosi, G., Fagiolo, G., Roventini, A.: Schumpeter meeting Keynes: A policy-friendly model of endogenous growth and business cycles. Journal of Economic Dynamics and Control 34, 1748–1767 (2010)CrossRefGoogle Scholar
  11. 11.
    Edmonds, B., Hales, D.: Replication, Replication and Replication: Some Hard Lessons from Model Alignment. Journal of Artificial Societies and Social Simulation 6 (2003),
  12. 12.
    Farmer, J., Foley, D.: The economy needs agent-based modelling. Nature 460, 685–686 (2009)CrossRefGoogle Scholar
  13. 13.
    Gallegati, M., Richiardi, M.: Agent Based Models in Economics and Complexity. In: Meyers, R. (ed.) Encyclopedia of Complexity and Systems Science, pp. 200–223. Springer Science (2009)Google Scholar
  14. 14.
    Gilbert, N., Troitzsch, K.G.: Simulation for the Social Scientist, 2nd edn. Open University Press, Berkshire (2005)Google Scholar
  15. 15.
    Gilbert, N.: Agent-Based Models. SAGE Publications (2008)Google Scholar
  16. 16.
    Grimm, V., Berger, U., Bastiansen, F., Eliassen, S., Ginot, V., Giske, J., Goss-Custard, J., Grand, T., Heinz, S., Huse, G., Huth, A., Jepsen, J., Jorgensen, C., Mooij, W., Muller, B., Peer, G., Piou, C., Railsback, S., Robbins, A., Robbins, M., Rossmanith, E., Ruger, N., Strand, E., Souissi, S., Stillman, R., Vabo, R., Visser, U., DeAngelis, D.: A standard protocol for describing individual-based and agent-based models. Ecological Modelling 198, 115–126 (2006)CrossRefGoogle Scholar
  17. 17.
    Grimm, V., Berger, U., DeAngelis, D.L., Polhill, G., Giske, J., Railsback, S.F.: The ODD protocol: a review and first update. Ecological Modeling 221, 2760–2768 (2010)CrossRefGoogle Scholar
  18. 18.
    Hayek, F.A.: Individualism and economic order. University of Chicago Press, Chicago (1948)Google Scholar
  19. 19.
    Holland, J.: Hidden Order: How Adaptation Builds Complexity. Addison-Wesley, Reading (1995)Google Scholar
  20. 20.
    Kaminski, B.: Podejście wieloagentowe do modelowania rynków: metody i zastosowania, Oficyna Wydawnicza SGH (2012)Google Scholar
  21. 21.
    Larman, C.: Iterative and Incremental Development: A Brief History. Computer 36(6), 47–56 (2003)CrossRefGoogle Scholar
  22. 22.
    Law, A.: Simulation Modeling and Analysis. McGraw-Hill (2006)Google Scholar
  23. 23.
    Leombruni, R.: Why are economists sceptical about agent-based simulations. Physica A 355, 103–109 (2005)CrossRefGoogle Scholar
  24. 24.
    Levy, M.: Agent Based Computational Economics. In: Meyers, R. (ed.) Encyclopedia of Complexity and Systems Science, pp. 92–112. Springer (2009)Google Scholar
  25. 25.
    Macal, C.M., North, M.J.: Tutorial on agent-based modeling and simulation part 2: how to model with agents. In: Perrone, L.F., Wieland, F.P., Liu, J., Lawson, B.G., Nicol, D.M., Fujimoto, R.M. (eds.) Proceedings of the 2006 Winter Simulation Conference (2006)Google Scholar
  26. 26.
    Miller, J.H., Page, S.E.: Complex Adaptive Systems. Princeton University Press (2007)Google Scholar
  27. 27.
    Moss, S.: Agent Based Modeling and Neoclassical Economics: A Critical Perspective. In: Meyers, R. (ed.) Encyclopedia of Complexity and Systems Science, pp. 176–184. Springer (2009)Google Scholar
  28. 28.
    NetLogo: NetLogo BehaviorSpace Guide, (2013), (accessed on May 10, 2013)
  29. 29.
    Object Management Group, OMG Unified Modeling Language (OMG UML), Superstructure. Version 2.4.1 (2011)Google Scholar
  30. 30.
    Oechslein, C., Klügl, F., Herrler, R., Puppe, F.: UML for Behavior-Oriented Multi-Agent Simulations. In: Dunin-Keplicz, B., Nawarecki, E. (eds.) CEEMAS 2001. LNCS (LNAI), vol. 2296, pp. 217–226. Springer, Heidelberg (2002)CrossRefGoogle Scholar
  31. 31.
    Olsen, M.: The logic of collective action. Harvard University Press, Cambridge (1965)Google Scholar
  32. 32.
    Palmer, R.G., Arthur, W.B., Holland, J.H., LeBaron, B., Tayler, P.: Artificial economic life: a simple model of a stockmarket. Physica D 75, 264–274 (1994)CrossRefGoogle Scholar
  33. 33.
    Philips, A.: Mechanical Models in Economic Dynamics. Economica 17, 283–305 (1950)CrossRefGoogle Scholar
  34. 34.
    Poutakidis, D., Winikoff, M., Padgham, L., Zhang, Z.: Debugging and Testing of Multi-Agent Systems using Design Artefacts. In: Bordini, R.H., et al. (eds.) Multi-Agent Programming. Springer (2009)Google Scholar
  35. 35.
    Railsback, S.F., Lytinen, S.L., Jackson, S.K.: Agent-based Simulation Platforms: Review and Development Recommendations. Simulation 82, 609–623 (2006)CrossRefGoogle Scholar
  36. 36.
    Richiardi, M.: Agent-based computational economics: a short introduction. The Knowledge Engineering Review 27, 137–149 (2012)CrossRefGoogle Scholar
  37. 37.
    Richiardi, M., Leombruni, R., Saam, N., Sonnesa, M.: A Common Protocol for Agent Based Social Simulation. Journal of Artificial Societies and Social Simulation 9 (2006)Google Scholar
  38. 38.
    Riolo, R.L., Cohen, M.D., Axelrod, R.: Evolution of cooperation without reciprocity. Nature 411, 441–443 (2001)CrossRefGoogle Scholar
  39. 39.
    Roussopoulos, N., Karagiannis, D.: Conceptual Modeling: Past, Present and the Continuum of the Future. In: Borgida, A.T., Chaudhri, V.K., Giorgini, P., Yu, E.S. (eds.) Conceptual Modeling: Foundations and Applications. LNCS, vol. 5600, pp. 139–152. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  40. 40.
    Santos, I.R., Santos, P.R.: Simulation Metamodels for Modeling Output Distribution Parameters. In: Proceedings of the 2007 Winter Simulation Conference, pp. 910–918 (2007)Google Scholar
  41. 41.
    Schelling, T.C.: Micromotives and macrobehavior, New York (1978)Google Scholar
  42. 42.
    Schroeder, R.G.: Operations Management-Decision Making in Operations Function, 4th edn. Mc Graw Hill International Editions, New York (1993)Google Scholar
  43. 43.
    Szufel, P.: O kosztowej efektywnosci procesow edukacyjnych, PhD Dissertation, SGH (2012)Google Scholar
  44. 44.
    Tesfatsion, L.: Agent-Based Computational Economics: Growing Economies From the Bottom Up. Artificial Life 8(1), 55–82 (2002)CrossRefGoogle Scholar
  45. 45.
    Tesfatsion, L., Judd, K. (eds.): Handbook of Computational Economics: Agent-Based Computational Economics. North-Holland (2006)Google Scholar
  46. 46.
    Thiele, J.C., Kurth, W., Grimm, V.: RNetLogo: an R package for running and exploring individual-based models implemented in NetLogo. In: Methods in Ecology and Evolution. British Ecological Society (2012); Early PreviewGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Bogumił Kamiński
    • 1
  • Przemysław Szufel
    • 1
  1. 1.Decision Support and Analysis DivisionWarsaw School of EconomicsPoland

Personalised recommendations