Skip to main content
SpringerLink
Log in

Uncertainty Assessment in Agent-Based Simulation: An Exploratory Study

  • Conference paper
  • First Online:
Autonomous Agents and Multiagent Systems (AAMAS 2017)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10642))

Included in the following conference series:

Abstract

This paper presents an overview of uncertainty assessment in agent-based simulations, mainly related to land use and cover change. Almost every multiagent-based simulation review has expressed the need for statistical methods to evaluate the certainty of the results. Yet these problems continue to be underestimated and often neglected. This work aims to review how uncertainty is being portrayed in agent-based simulation and to perform an exploratory study to use statistical methods to estimate uncertainty. MASE, a Multi-Agent System for Environmental simulation, is the system under study. We first identified the most sensitive parameters using Morris One-at-a-Time sensitivity analysis. The efforts to assess agent-based simulation through statistical methods are paramount to corroborate and improve the level of confidence of the research that has been made in land use simulation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://dakota.sandia.gov/.

  2. 2.

    http://www.uq-pyl.com/.

  3. 3.

    http://meme-suite.org/.

  4. 4.

    Software Availability: http://mase.cic.unb.br/.

References

  1. Albrecht, S.V., Ramamoorthy, S.: Are you doing what i think you are doing? criticising uncertain agent models. In: Proceedings of the 31st Conference on Uncertainty in Artificial Intelligence, Amsterdam, Netherlands, p. 10 (2015)

    Google Scholar 

  2. Herd, B., Miles, S., McBurney, P., Luck, M.: MC \(^{2}\) MABS: a monte carlo model checker for multiagent-based simulations. In: Gaudou, B., Sichman, J.S. (eds.) MABS 2015. LNCS (LNAI), vol. 9568, pp. 37–54. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-31447-1_3

    Chapter  Google Scholar 

  3. Bommel, P.: Foreword. In: Adamatti, D.F. (ed.) Multi-Agent Based Simulations Applied to Biological and Environmental Systems, pp. xv-xviii. IGI Global, Hershey (2017)

    Google Scholar 

  4. Coelho, C.C.G., Abreu, C.G., Ramos, R.M., Mendes, A.H.D., Teodoro, G., Ralha, C.G.: MASE-BDI: agent-based simulator for environmental land change with efficient and parallel auto-tuning. Appl. Intell. 45(3), 904–922 (2016)

    Article  Google Scholar 

  5. Campolongo, F., Braddock, R.: The use of graph theory in the sensitivity analysis of the model output: a second order screening method. Reliab. Eng. Syst. Saf. 64(1), 1–12 (1999), https://doi.org/10.1016/S0951-8320(98)00008-8, http://linkinghub.elsevier.com/retrieve/pii/S0951832098000088

  6. Casti, J.L.: Complexification: Explaining a Paradoxical World through the Science of Surprise, reprint edn. HarperCollins (1995)

    Google Scholar 

  7. Intergovernmental Panel on Climate Change: Climate Change 2013 - The Physical Science Basis: Working Group I Contribution to the Fifth Assessment Report of the Intergovernmental Panel on Climate Change. Cambridge University Press, Cambridge (2014)

    Google Scholar 

  8. Gan, Y., Duan, Q., Gong, W., Tong, C., Sun, Y., Chu, W., Ye, A., Miao, C., Di, Z.: A comprehensive evaluation of various sensitivity analysis methods: a case study with a hydrological model. Environ. Model. Softw. 51, 269–285 (2014)

    Article  Google Scholar 

  9. Goldsman, D., Tokol, G.: Output analysis procedures for computer simulations. In: Joines, J., Barton, R.R., Kang, K., Fishwick, P. (eds.) Proceedings of the 2000 Winter Simulation Conference, pp. 39–45 (2000)

    Google Scholar 

  10. Grimm, V., Berger, U., Bastiansen, F., Eliassen, S., Ginot, V., Giske, J., Goss-Custard, J., Grand, T., Heinz, S.K., Huse, G.: A standard protocol for describing individual-based and agent-based models. Ecol. Model. 198(1–2), 115–126 (2006), http://linkinghub.elsevier.com/retrieve/pii/S0304380006002043

  11. Grimm, V., Berger, U., DeAngelis, D.L., Polhill, J.G., Giske, J., Railsback, S.F.: The ODD protocol: a review and first update. Ecol. Model. 221(23), 2760–2768 (2010), http://linkinghub.elsevier.com/retrieve/pii/S030438001000414X

  12. Heath, B., Hill, R., Ciarallo, F.: A survey of agent-based modeling practices (January 1998 to July 2008). JASSS 12(4), 1–49 (2009)

    Google Scholar 

  13. Houghton, J., Filho, L.M., Callander, B., Harris, N., Kattenberg, A., Maskell, K. (eds.): Climate Change 1995 The Science of Climate Change. The Intergovernmental Panel on Climate Change (1996)

    Google Scholar 

  14. Iooss, B., Lemaître, P.: A review on global sensitivity analysis methods. In: Dellino, G., Meloni, C. (eds.) Uncertainty Management in Simulation-Optimization of Complex Systems. ORSIS, vol. 59, pp. 101–122. Springer, Boston (2015). https://doi.org/10.1007/978-1-4899-7547-8_5

    Chapter  Google Scholar 

  15. Kelly (Letcher), R.A., Jakeman, A.J., Barreteau, O., Borsuk, M.E., ElSawah, S., Hamilton, S.H., Henriksen, H.J., Kuikka, S., Maier, H.R., Rizzoli, A.E., van Delden, H., Voinov, A.A.: Selecting among five common modelling approaches for integrated environmental assessment and management. Environ. Model. Softw. 47, 159–181 (2013)

    Google Scholar 

  16. Kleijnen, J.P., Sanchez, S.M., Lucas, T.W., Cioppa, T.M.: A user’s guide to the brave new world of designing simulation experiments. INFORMS J. Comput. 17(3), 263–289 (2005), https://harvest.nps.edu/papers/UserGuideSimExpts.pdf

  17. Le, Q.B., Seidl, R., Scholz, R.W.: Feedback loops and types of adaptation in the modelling of land-use decisions in an agent-based simulation. Environ. Model. Softw. 27–28, 83–96 (2012)

    Article  Google Scholar 

  18. Lee, J.S., Filatova, T., Ligmann-Zielinska, A., Hassani-Mahmooei, B., Stonedahl, F., Lorscheid, I., Voinov, A., Polhill, G., Sun, Z., Parker, D.C.: The complexities of agent-based modeling output analysis. JASSS 18(4), 1–25 (2015)

    Article  Google Scholar 

  19. Levy, S., Steinberg, D.M.: Computer experiments: a review. AStA Adv. Stat. Anal. 94(4), 311–324 (2010)

    Article  MathSciNet  Google Scholar 

  20. Li, J.D., Duan, Q.Y., Gong, W., Ye, A.Z., Dai, Y.J., Miao, C.Y., Di, Z.H., Tong, C., Sun, Y.W.: Assessing parameter importance of the common land model based on qualitative and quantitative sensitivity analysis. Hydrol. Earth Syst. Sci. Discuss. 10(2), 2243–2286 (2013)

    Article  Google Scholar 

  21. Lorscheid, I., Heine, B.O., Meyer, M.: Opening the ‘black box’ of simulations: increased transparency and effective communication through the systematic design of experiments. Comput. Math. Organ. Theory 18(1), 22–62 (2012)

    Article  Google Scholar 

  22. Marks, R.E.: Validating simulation models: a general framework and four applied examples. Comput. Econ. 30(3), 265–290 (2007)

    Article  MATH  Google Scholar 

  23. Mastrandrea, M.D., Field, C.B., Stocker, T.F., Edenhofer, O., Ebi, K.L., Frame, D.J., Held, H., Kriegler, E., Mach, K.J., Matschoss, P.R., Plattner, G.K., Yohe, G.W., Zwiers, F.W.: Guidance note for lead authors of the ipcc fifth assessment report on consistent treatment of uncertainties. In: Intergovernmental Panel on Climate Change (IPCC), pp. 1–7 (2010)

    Google Scholar 

  24. McKay, M.D., Morrison, J.D., Upton, S.C.: Evaluating prediction uncertainty in simulation models. Comput. Phys. Commun. 117(1–2), 44–51 (1999)

    Article  MATH  Google Scholar 

  25. Morris, M.D.: Factorial sampling plans for preliminary computational experiments. Technometrics 33(2), 161–174 (1991)

    Article  Google Scholar 

  26. Paegelow, M., Camacho Olmedo, M.T., Mas, J.F., Houet, T.: Benchmarking of LUCC modelling tools by various validation techniques and error analysis. Cybergeo 701(online), 29 (2014)

    Google Scholar 

  27. Pontius, R.G., Boersma, W., Castella, J.C., Clarke, K., Nijs, T., Dietzel, C., Duan, Z., Fotsing, E., Goldstein, N., Kok, K., Koomen, E., Lippitt, C.D., McConnell, W., Mohd Sood, A., Pijanowski, B., Pithadia, S., Sweeney, S., Trung, T.N., Veldkamp, A.T., Verburg, P.H.: Comparing the input, output, and validation maps for several models of land change. Ann. Reg. Sci. 42(1), 11–37 (2008)

    Article  Google Scholar 

  28. Ralha, C.G., Abreu, C.G.: A multi-agent-based environmental simulator. In: Adamatti, D.F. (ed.) Multi-Agent Based Simulations Applied to Biological and Environmental Systems, chap. 5, pp. 106–127. IGI Global, Hershey (2017)

    Google Scholar 

  29. Ralha, C.G., Abreu, C.G., Coelho, C.G., Zaghetto, A., Macchiavello, B., Machado, R.B.: A multi-agent model system for land-use change simulation. Environ. Model. Softw. 42, 30–46 (2013)

    Article  Google Scholar 

  30. Saltelli, A., Ratto, M., Andres, T., Campolongo, F., Cariboni, J., Gatelli, D., Saisana, M., Tarantola, S.: Global Sensitivity Analysis: The Primer. Wiley (2008)

    Google Scholar 

  31. Smajgl, A., Brown, D.G., Valbuena, D., Huigen, M.G.A.: Environmental modelling & software empirical characterisation of agent behaviours in socio-ecological systems. Environ. Model. Softw. 26(7), 837–844 (2011), https://doi.org/10.1016/j.envsoft.2011.02.011

  32. Tong, C.: PSUADE Short Manual (Version 1.7). Lawrence Livermore National Laboratory (LLNL), Livermore, CA (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Brasília (UnB), Brasília, DF, Brazil

    Carolina G. Abreu & Célia G. Ralha

Authors
  1. Carolina G. Abreu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Célia G. Ralha
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carolina G. Abreu .

Editor information

Editors and Affiliations

  1. University of Central Florida, Orlando, Florida, USA

    Gita Sukthankar

  2. IIIA-CSIC, Bellaterra, Spain

    Juan A. Rodriguez-Aguilar

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Abreu, C.G., Ralha, C.G. (2017). Uncertainty Assessment in Agent-Based Simulation: An Exploratory Study. In: Sukthankar, G., Rodriguez-Aguilar, J. (eds) Autonomous Agents and Multiagent Systems. AAMAS 2017. Lecture Notes in Computer Science(), vol 10642. Springer, Cham. https://doi.org/10.1007/978-3-319-71682-4_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-71682-4_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-71681-7

  • Online ISBN: 978-3-319-71682-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation