Validation of Spatial Microsimulation Models

Abstract

Spatial microsimulation models, both static and dynamic, are a useful means to estimate area-level data, whatever these data are regarding, be it health, socio-economic status or income/finance. However, in order for planners and government to be able to use and rely on these data, it is essential that the modellers can show that the estimates are an accurate presentation of the real world and are reliable. Generally, to verify the integrity of any model, it is necessary to validate the model outputs, using both internal and external validation methods. However, for spatial microsimulation models, validation is a massive challenge. This is because, generally, these models are used to estimate data that does not otherwise exist, perhaps due to confidentiality reasons (e.g. income or medical data for individuals) and/or because it would be expensive and time consuming to try to collect a large sample of data for the population in question (particularly as, in many countries, national sample datasets already exist, thus it would also be a duplication of both time and money).

In this chapter, different methods of validation for both static and dynamic spatial microsimulation models are discussed, including indication of acceptable tolerances for validation measures. We also examine reasons why validation may be poor. We conclude that researchers need to be transparent about their validation methodologies and realistic about the strength of the estimates and accuracy of the models. Working with our colleagues from other disciplines may help us to improve validation methodologies, and working with local authorities and government, in lay language, may increase acceptability of these models.