Experimentation and Output Analysis

Abstract

In this chapter we explore the experimentation and output analysis phases of a modelling and simulation project which are both central to the success of the project. In other words, we examine the process of correctly formulating and carrying out goal-directed experiments with the simulation program and then extracting meaningful information from the data acquired via its output variables. Our interest throughout this chapter is primarily with scalar output variables SOVs which include the derived scalar output variable (whose value is derived from recorded values of a discrete-time variable) and the simple scalar output variable (whose value is calculated on the fly). Also of interest are the analysis techniques applied to these SOVs in achieving the project goals. An SOV acquires a value from a simulation run (the execution of the simulation model over the observation interval). Since the SUI has random behaviour, the SOV is typically a random variable. Numerous runs are required to give a set of values from which can be computed a point estimate (i.e. an estimate of the mean of the SOVs distribution) and a confidence interval which determines how close the point estimate is to the true mean. Thus the confidence interval allows the experimenter to determine the reliability of the point estimate. Techniques for determining the point estimate and confidence intervals are presented for both the bounded horizon study (where the right-hand side of the observation interval is fixed and where the SUI typically contains transient stochastic processes) and the steady-state study (where the right-hand side of the observation interval is not fixed and the SUI has a steady-state behaviour). The chapter also adapts these statistical techniques for comparison of alternative behaviours of the SUI when model parameters are varied.