Skip to main content
SpringerLink
Log in

Model Calibration

  • Chapter
  • First Online:
Advances in Modeling and Simulation

Part of the book series: Simulation Foundations, Methods and Applications ((SFMA))

Abstract

Simulation model calibration refers to the iterative process of comparing the outputs of a simulation model with the observed quantities in the real system, and making changes to model input parameters accordingly to achieve an acceptable level of agreement between the simulation model and the real system. While calibration in a broader context may involve structural changes to the simulation model, this chapter focuses on the calibration of simulation model parameters that cannot be accurately estimated or specified for various reasons. When the simulation is time-consuming, has significant noise, and/or has a large number of parameters to calibrate, automatic and efficient calibration methods are critical to the success of any simulation-based analysis and optimization . This chapter discusses two main categories of general calibration methods: (1) direct calibration methods that search for the optimal calibration parameter that minimizes the difference between real system observations and simulation model outputs; and (2) Bayesian calibration methods that combine real system observations with prior knowledge to obtain a posterior distribution on the calibration parameters.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 16.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Absi GN, Mahadevan S (2016) Multi-fidelity approach to dynamics model calibration. Mech Syst Signal Proces 68:189–206

    Article  Google Scholar 

  • Ankenman B, Nelson BL, Staum J (2010) Stochastic kriging for simulation metamodeling. Oper Res 58:371–382

    Article  MathSciNet  MATH  Google Scholar 

  • Aral KD, Chick SE, Grabosch A (2014) Primary preventive care model for type 2 diabetes: input calibration with response data. In: Proceedings of the winter simulation conference. IEEE, Piscataway, NJ, pp 1399–1410

    Google Scholar 

  • Barton RR (2009) Simulation optimization using metamodels. In: Proceedings of the winter simulation conference. IEEE, Piscataway, NJ, pp 230–238

    Google Scholar 

  • Chen X, Ankenman BE, Nelson BL (2012) The effects of common random numbers on stochastic kriging metamodels. ACM Trans Model Comput Simul 22:7

    MathSciNet  Google Scholar 

  • Chen X, Ankenman BE, Nelson BL (2013) Enhancing stochastic kriging metamodels with gradient estimators. Oper Res 61:512–528

    Article  MathSciNet  MATH  Google Scholar 

  • Chen W, Gao S, Chen CH, Shi L (2014) An optimal sample allocation strategy for partition-based random search. IEEE Trans Autom Sci Eng 11:177–186

    Article  Google Scholar 

  • Flötteröd G, Bierlaire M, Nagel K (2011) Bayesian demand calibration for dynamic traffic simulations. Transp Sci 45:541–561

    Article  Google Scholar 

  • Frazier P, Powell WB, Simão HP (2009) Simulation model calibration with correlated knowledge-gradients. In: Proceedings of the 2009 winter simulation conference. IEEE, Piscataway, NJ, pp 339–351

    Google Scholar 

  • Fu MC (2015a) Stochastic gradient estimation. In: Handbook of simulation optimization. Springer, New York, pp 105–147

    Google Scholar 

  • Fu MC (2015b) Handbook of simulation optimization. Springer, New York

    Google Scholar 

  • Han G, Santner TJ, Rawlinson JJ (2009) Simultaneous determination of tuning and calibration parameters for computer experiments. Technometrics 51:464–474

    Article  MathSciNet  Google Scholar 

  • He D, Lee LH, Chen CH, Fu MC, Wasserkrug S (2010) Simulation optimization using the cross-entropy method with optimal computing budget allocation. ACM Trans Model Comput Simul 20:4

    Article  Google Scholar 

  • Henclewood D, Suh W, Rodgers M, Hunter M, Fujimoto R (2012) A case for real-time calibration of data-driven microscopic traffic simulation tools. In: Proceedings of the winter simulation conference. IEEE, Piscataway, NJ, pp 1670–1681

    Google Scholar 

  • Henderson DA, Boys RJ, Krishnan KJ, Lawless C, Wilkinson DJ (2009) Bayesian emulation and calibration of a stochastic computer model of mitochondrial DNA deletions in substantia nigra neurons. J Am Stat Assoc 104:76–87

    Article  MathSciNet  MATH  Google Scholar 

  • Hong LJ, Nelson BL, Xu J (2010) Speeding up COMPASS for high-dimensional discrete optimization via simulation. Oper Res Lett 38:550–555

    Article  MATH  Google Scholar 

  • Hong LJ, Nelson BL, Xu J (2015) Discrete optimization via simulation. In: Handbook of simulation optimization. Springer, New York, pp 9–44

    Google Scholar 

  • Johnson RT, Lampe TA, Seichter S (2009) Calibration of an agent-based simulation model depicting a refugee camp scenario. In: Proceedings of the winter simulation conference. IEEE, Piscataway, NJ, pp 1778–1786

    Google Scholar 

  • Kennedy MC, O’Hagan A (2001) Bayesian calibration of computer models. J R Stat Soc Ser B 63:425–464

    Article  MathSciNet  MATH  Google Scholar 

  • Kushner HJ, Yin G (2003) Stochastic approximation and recursive algorithms and applications. Springer, New York

    MATH  Google Scholar 

  • Latek MM, Mussavi Rizi SM, Geller A (2013) Verification through calibration: an approach and a case study of a model of conflict in Syria. In: Proceedings of the winter simulation conference. IEEE, Piscataway, NJ, pp 1649–1660

    Google Scholar 

  • Leathwick DM (2013) Managing anthelmintic resistanceparasite fitness, drug use strategy and the potential for reversion towards susceptibility. Vet Parasitol 198:145–153

    Article  Google Scholar 

  • Leathwick DM, Hosking BC (2009) Managing anthelmintic resistance: modelling strategic use of a new anthelmintic class to slow the development of resistance to existing classes. NZ Vet J 57:203–207

    Article  Google Scholar 

  • Loeppky JL, Sacks J, Welch WJ (2009) Choosing the sample size of a computer experiment: a practical guide. Technometrics 51:366–376

    Article  MathSciNet  Google Scholar 

  • Mackinnon MJ (2005) Drug resistance models for malaria. Acta Tropica 94:207–217

    Article  Google Scholar 

  • Matus O, Barrera J, Moreno E, Rubino G (2016) Calibrating a dependent failure model for computing reliabilities on telecommunication networks. In: Proceedings of the winter simulation conference. IEEE, Piscataway, NJ, pp 490–500

    Google Scholar 

  • Molento MB, Nielsen MK, Kaplan RM (2012) Resistance to avermectin/milbemycin anthelmintics in equine cyathostominscurrent situation. Vet Parasitol 185:16–24

    Article  Google Scholar 

  • Nielsen MK, Vidyashankar AN, Hanlon BM, Diao G, Petersen SL, Kaplan RM (2013) Hierarchical model for evaluating pyrantel efficacy against strongyle parasites in horses. Vet Parasitol 197:614–622

    Article  Google Scholar 

  • Rawlinson JJ, Furman BD, Li S, Wright TM, Bartel DL (2006) Retrieval, experimental, and computational assessment of the performance of total knee replacements. J Orthop Res 24:1384–1394

    Article  Google Scholar 

  • Salemi P, Nelson BL, Staum J (2014) Discrete optimization via simulation using Gaussian Markov random fields. In: Proceedings of the winter simulation conference. IEEE, Piscataway, NJ, pp 3809–3820

    Google Scholar 

  • Santner TJ, Williams BJ, Notz WI (2013) The design and analysis of computer experiments. Springer, New York

    MATH  Google Scholar 

  • Shi D, Brooks RJ (2007) The range of predictions for calibrated agent-based simulation models. In: Proceedings of the winter simulation conference. IEEE, Piscataway, NJ, pp 1198–1206

    Google Scholar 

  • Shi L, Olafsson S (2009) Nested partitions method, theory and applications. Springer, New York

    MATH  Google Scholar 

  • Taghiyeh S, Xu J (2016) A new particle swarm optimization algorithm for noisy optimization problems. Swarm Intell 10:161–192

    Article  Google Scholar 

  • Vidyashankar AN, Xu J (2015) Stochastic optimization using Hellinger distance. In: Proceedings of the winter simulation conference. IEEE, Piscataway, NJ, pp 3702–3713

    Google Scholar 

  • Vock S, Enz S, Cleophas C (2014) Genetic algorithms for calibrating airline revenue management simulations. In: Proceedings of the winter simulation conference. IEEE, Piscataway, NJ, pp 264–275

    Google Scholar 

  • Wang H, Pasupathy R, Schmeiser BW (2013) Integer-ordered simulation optimization using R-SPLINE: retrospective search with piecewise-linear interpolation and neighborhood enumeration. ACM Trans Model Comput Simul 23:17

    Article  MathSciNet  Google Scholar 

  • Xu J (2012) Efficient discrete optimization via simulation using stochastic kriging. In: Proceedings of the winter simulation conference. IEEE, Piscataway, NJ, pp 466–477

    Google Scholar 

  • Xu J, Nelson BL, Hong, LJ (2010) Industrial strength COMPASS: a comprehensive algorithm and software for optimization via simulation. ACM Trans Model Comput Simul 20:3:1–3:29

    Google Scholar 

  • Xu J, Nelson BL, Hong LJ (2013) An adaptive hyperbox algorithm for high-dimensional discrete optimization via simulation problems. INFORMS J Comput 25:133–146

    Article  MathSciNet  Google Scholar 

  • Xu J, Vidyashankar A, Nielsen, MK (2014) Drug resistance or re-emergence? Simulating equine parasites. ACM Trans Model Comput Simul 24:20:1–20:23

    Google Scholar 

  • Xu J, Huang E, Chen CH, Lee, LH (2015) Simulation optimization: a review and exploration in the new era of cloud computing and big data. Asia-Pac J Oper Res 32:1650017:1–1650017:26

    Google Scholar 

  • Xu J, Huang E, Hsieh L, Lee LH, Jia QS, Chen CH (2016a) Simulation optimization in the era of industrial 4.0 and the industrial internet. J Simul 10:310–320

    Google Scholar 

  • Xu J, Zhang S, Huang E, Chen CH, Lee LH, Celik N (2016b) MO\(^2\)TOS: Multi-fidelity optimization with ordinal transformation and optimal sampling. Asia-Pac J Oper Res 33:1650017

    Google Scholar 

  • Xu J, Zhang S, Huang E, Chen CH, Lee LH, Celik N (2014) Efficient multi-fidelity simulation optimization. In: Proceedings of the winter simulation conference. IEEE, Piscataway, NJ, pp 3940–3951

    Google Scholar 

  • Yuan J, Ng SH (2013a) An entropy based sequential calibration approach for stochastic computer models. In: Proceedings of the winter simulation conference. IEEE, Piscataway, NJ, pp 589–600

    Google Scholar 

  • Yuan J, Ng SH (2013b) A sequential approach for stochastic computer model calibration and prediction. Reliab Eng Syst Saf 111:273–286

    Google Scholar 

  • Yuan J, Ng SH, Tsui KL (2013) Calibration of stochastic computer models using stochastic approximation methods. IEEE Trans Autom Sci Eng 10:171–186

    Article  Google Scholar 

  • Yuan J, Ng SH (2015) Calibration, validation, and prediction in random simulation models: Gaussian process metamodels and a Bayesian integrated solution. ACM Trans Model Comput Simul 25:18:1–18:25

    Google Scholar 

  • Zhang S, Lee LH, Chew EP, Xu J, Chen CH (2016) A simulation budget allocation procedure for enhancing the efficiency of optimal subset selection. IEEE Trans Autom Control 61:62–75

    Article  MathSciNet  MATH  Google Scholar 

  • Zhang S, Xu J, Lee LH, Chew EP, Wong WP, Chen CH (2017) Optimal computing budget allocation for particle swarm optimization in stochastic optimization. IEEE Trans Evol Comput 21:206–219

    Article  Google Scholar 

  • Zhu C, Xu J, Chen CH, Lee LH, Hu JQ (2016) Balancing search and estimation in random search based stochastic simulation optimization. IEEE Trans Autom Control 61:3593–3598

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work has been supported in part by National Science Foundation under Award CMMI-1233376, CMMI-1462787, and ECCS-1462409 (jointly funded by United States Air Force Office of Scientific Research).

Author information

Authors and Affiliations

  1. George Mason University, 4400 University Drive, MS 4A6, Fairfax, VA, 22030, USA

    Jie Xu

Authors
  1. Jie Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jie Xu .

Editor information

Editors and Affiliations

  1. The MITRE Corporation, Hampton, Virginia, USA

    Andreas Tolk

  2. Department of Supply Chain Management, Arizona State University, Tempe, Arizona, USA

    John Fowler

  3. National Institute of Standards and Technology, Gaithersburg, Maryland, USA

    Guodong Shao

  4. T&OM Area, INSEAD, Fontainebleau, France

    Enver Yücesan

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG (outside the USA)

About this chapter

Cite this chapter

Xu, J. (2017). Model Calibration. In: Tolk, A., Fowler, J., Shao, G., Yücesan, E. (eds) Advances in Modeling and Simulation. Simulation Foundations, Methods and Applications. Springer, Cham. https://doi.org/10.1007/978-3-319-64182-9_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-64182-9_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-64181-2

  • Online ISBN: 978-3-319-64182-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation