Experimentation and Output Analysis

  • Louis G. Birta
  • Gilbert Arbez
Part of the Simulation Foundations, Methods and Applications book series (SFMA)


In this chapter we explore the experimentation and output analysis phases of a modelling and simulation project which are both central to the success of the project. In other words, we examine the process of correctly formulating and carrying out goal-directed experiments with the simulation program and then extracting meaningful information from the data acquired via its output variables. Our interest throughout this chapter is primarily with scalar output variables SOVs which include the derived scalar output variable (whose value is derived from recorded values of a discrete-time variable) and the simple scalar output variable (whose value is calculated on the fly). Also of interest are the analysis techniques applied to these SOVs in achieving the project goals. An SOV acquires a value from a simulation run (the execution of the simulation model over the observation interval). Since the SUI has random behaviour, the SOV is typically a random variable. Numerous runs are required to give a set of values from which can be computed a point estimate (i.e. an estimate of the mean of the SOVs distribution) and a confidence interval which determines how close the point estimate is to the true mean. Thus the confidence interval allows the experimenter to determine the reliability of the point estimate. Techniques for determining the point estimate and confidence intervals are presented for both the bounded horizon study (where the right-hand side of the observation interval is fixed and where the SUI typically contains transient stochastic processes) and the steady-state study (where the right-hand side of the observation interval is not fixed and the SUI has a steady-state behaviour). The chapter also adapts these statistical techniques for comparison of alternative behaviours of the SUI when model parameters are varied.


Point Estimate Simulation Program Observation Interval Project Goal Time Cell 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


  1. 1.
    Banks J, Carson JS II, Nelson BL, Nicol DM (2005) Discrete-event system simulation, 4th edn. Pearson Prentice Hall, Upper Saddle RiverGoogle Scholar
  2. 2.
    Goldsman D, Nelson BL (1998) Comparing systems via simulation. In: Banks J (ed) Handbook of simulation. Wiley, New York, pp 273–306CrossRefGoogle Scholar
  3. 3.
    Goldsman D, Nelson BL (2001) Statistical selection of the best system. In: Peters BA, Smith JS, Medeiros DJ, Rohrer MW (eds) Proceeding of the 2001 winter simulation Conference. IEEE Press, Piscataway, pp 139–146Google Scholar
  4. 4.
    Goldsman D, Schruben LW, Swain JJ (1994) Test for transient means in simulation time series. Nav Res Logist Q 41:171–187CrossRefzbMATHGoogle Scholar
  5. 5.
    Law AM, Kelton DW (2000) Simulation modeling and analysis, 3rd edn. McGraw-Hill, New YorkGoogle Scholar
  6. 6.
    Robinson S (2002) A statistical process control approach for estimating the warm-up period. In: Proceeding of the 2002 winter simulation conference. IEEE, Piscataway, pp 439–446Google Scholar
  7. 7.
    Roth E (1994) The relaxation time heuristic for the initial transient problem in M/M/K queuing systems. Eur J Oper Res 72:376–386CrossRefzbMATHGoogle Scholar
  8. 8.
    Welch P (1983) The statistical analysis of simulation results. In: Lavenberg S (ed) The computer performance modeling handbook. Academic, New York, pp 268–328Google Scholar

Copyright information

© Springer-Verlag London 2013

Authors and Affiliations

  • Louis G. Birta
    • 1
  • Gilbert Arbez
    • 1
  1. 1.School of Electrical Engineering and Computer ScienceUniversity of OttawaOttawaCanada

Personalised recommendations