Abstract
While Thomas Kuhn’s theory of scientific revolutions does not specifically deal with validation, the validation of simulations can be related in various ways to Kuhn’s theory: (1) Computer simulations are sometimes depicted as located between experiments and theoretical reasoning, thus potentially blurring the line between theory and empirical research. Does this require a new kind of research logic that is different from the classical paradigm which clearly distinguishes between theory and empirical observation? I argue that this is not the case. (2) Another typical feature of computer simulations is their being “motley” (Winsberg in Philos Sci 70:105–125, 2003) with respect to the various premises that enter into simulations. A possible consequence is that in case of failure it can become difficult to tell which of the premises is to blame. Could this issue be understood as fostering Kuhn’s mild relativism with respect to theory choice? I argue that there is no need to worry about relativism with respect to computer simulations, in particular. (3) The field of social simulations, in particular, still lacks a common understanding concerning the requirements of empirical validation of simulations. Does this mean that social simulations are still in a prescientific state in the sense of Kuhn? My conclusion is that despite ongoing efforts to promote quality standards in this field, lack of proper validation is still a problem of many published simulation studies and that, at least large parts of social simulations must be considered as prescientific.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
In the realm of computer simulations the term verification is, somewhat confusingly, reserved for checking wether the simulation software is free from programming errors (so-called “bugs”) and whether it is faithful to the mathematical model or theory on which it is based. The term validation is used for the empirical testing of the simulation’s results. See also Chap. 4 by Murray-Smith in this volume.
- 2.
See also Chap. 39 by Lenhard in this volume.
- 3.
Because theory-ladenness of observation is an often misunderstood topic, two remarks are in order: (1) Theory-ladenness of observation as such does not blur the distinction between theory and observation. At worst we have a distinction between pure theory (without any observational component) and theory-laden observation. (2) Theory-ladeness of observation does not lead to a vicious circle when confirming theories by empirical observation. This is true, as long as the observations are not laden with the particular theories for the confirmation of which they are used. There are areas in science where no sharp distinction between theoretical reasoning and reporting of observations is made. However, as far as computer simulations are concerned, it is clear that because Turing Machines do not make observations, a computer program is always a theoretical entity—not withstanding the fact that a computer program may represent an empirical setting or make use of empirical data. In the latter respect it can be compared with a physical theory that may in fact represent empirical reality as well as contain natural constants (i.e., empirical data).
- 4.
See also Chap. 37 by Beisbart in this volume.
- 5.
In simulation-science the term empirical is sometimes used to distinguish simulation and numerical methods from mathematical analysis (Phelps 2016 is an example of this.). But this is just a different use of words and should not be confused with “empirical” in the sense of being observation-based as the word is understood in the context of empirical science.
- 6.
See Arnold (2013, Sect. 3.4) for a case-study containing a detailed description of this hierarchy of premises.
- 7.
But see Lenhard in Chap. 39 in this volume, who paints a very different picture. I cannot resolve the differences here. In part they are due to Lenhard using examples where “due to interactivity, modularity does not break down a complex system into separately manageable pieces.” (Lenhard and Winsberg (2010), p. 256) To me it seems that as far as software design goes, it is always possible—and in fact good practice—to design the system in such a way that each unit can be tested separately. As far as validation goes, I admit that this may not work as easily because of restrictions concerning the availability of empirical data.
- 8.
There are scientists who deny even this and who also believe that without formal models no explanation of any sort is possible in history or social science. I am a bit at loss for giving proper references for this point of view, because I have mostly been confronted with it either in discussions with scientists or by anonymous referees of journals of analytic philosophy. The published source I know of that comes closest to this stance is the keynote “Why model?” by Epstein (2008), which I have discussed in Arnold (2014).
- 9.
This seems to be the standard case for applying the GCM in organizational science. See Fardal and Sornes (2008) and Delgoshaei and Fatahi (2013) for example. It will be interesting to see whether the more refined simulation models of the GCM that have been published more recently (Fioretti and Lomi 2008) will bring about an increased use of simulation models in applied studies referring to the GCM or not.
- 10.
This is precisely where Axelrod’s simulations was lacking, because (a) his tournament of reiterated Prisoner’s Dilemmas is too far removed from the phenomenology of either animal or human interaction to be prima facie plausible, and (b) his results were—unbeknownst to him—highly volatile with respect to the simulation setup and thus also lack plausibility.
- 11.
They discuss this under the heading of “theory-ladenness of observations”, though their examples suggest that the issue at stake is rather different interpretations of observations or a focus on different observations depending on the research questions than different observations due to a different theoretical background.
- 12.
A most notable initiative in this respect has been the introduction of the ODD Protocol for the standardized description of agent-based-models (Railsback and Grimm 2012).
References
Ahrweiler, P., & Gilbert, N. (2015). The quality of social simulation: An example from research policy modelling. In M. Janssen, M. A. Wimmer, & A. Deljoo (Eds.), Policy practice and digital science: Integrating complex systems, social simulation and public administration in policy research (pp. 35–55). Cham: Springer International Publishing. https://doi.org/10.1007/978-3-319-12784-2_3. ISBN: 978-3-319-12784-2.
Arnold, E. (2013). Experiments and simulations: Do they fuse? In E. Arnold, & J. Duran (Eds.), Computer simulations and the changing face of scientific experimentation (pp. 46–75). Newcastle: Cambridge Scholars Publishing. 978-1443847926.
Arnold, E. (2014). What’s wrong with social simulations? The Monist, 97(3), 361–379. https://doi.org/10.5840/monist201497323. ISSN: 0026-9662.
Axelrod, R. (1984). The evolution of cooperation. Basic Books.
Axtell, R. L., et al. (2002) Population growth and collapse in a multiagent model of the Kayenta Anasazi in Long House Valley. Proceedings of the National Academy of Sciences, 99(3), 7275–7279. https://doi.org/10.1073/pnas.092080799. ISSN: 0027-8424.
Beisbart, C. (2011). A transformation of normal science. Computer simulations from a philosophical perspective. unpublished.
Binmore, K. (1998) Game theory and the social contract II. Just playing. Cambridge, Massachusetts/London, England: MIT Press.
Binmore, K. (1994). Game theory and the social contract I. Playing fair. Fourth printing. Cambridge, Massachusetts/London, England: MIT Press.
Bird, A. (2013). Thomas Kuhn. In E. N. Zalta (Ed.), The stanford encyclopedia of philosophy. Fall 2013. Metaphysics Research Lab, Stanford University.
Cartwright, N. (1983). How the laws of physics lie.Clarendon paperbacks. Oxford University Press. Clarendon Press. ISBN: 9780198247043.
Carusi, A., Rodriguez, B., & Burrage, K. (2013). Model systems in computational systems biology. In E. Arnold, & J. Duran (Eds.), Computer simulations and the changing face of scientific experimentation. (Chap. 6).
Cohen, M. D., March, J. G., & Olsen, J. P. (1972). A grabage can model of organizational choide. Administrative Science Quarterly, 17, 1–25.
Delgoshaei, B., & Fatahi, M. (2013). Garbage can decision-making in a matrix structure. A Case study of linköping university. Linköping University/Department of Management and Engineering. urn:nbn:se:liu:diva-95612.
Dugatkin, L. A. (1997). Cooperation among animals. Oxford University Press.
Dupré, J. (1994). Against scientific imperialism. Philosophy of Science Association Proceedings, 2, 374–381.
Epstein, J. M. (2008). Why model? Journal of Artificial Societies and Social Simulation, 11(4), 12. ISSN: 1460-7425.
Epstein, J. M., & Axtell, R. L. (1996). Growing artificial societies: Social science from the bottom up. MIT Press.
Fardal, H., & Sørnes, J. O. (2008). Is strategic decision-making. A garbage can view. Issues in Informing Science and Information Technology, 5.
Feyerabend, P. (1975/1983). Wider den Methodenzwang. Suhrkamp Verlag.
Fioretti, G., & Lomi, A. (2008). An agent-based representation of the garbage can model of organizational choice. Journal of Artificial Societies and Social Simulation, 11(1), 1. ISSN: 1460-7425.
Gilbert, N., & Troitzsch, K. (2005). Simulation for the social scientist. New York: Open University Press.
Gleick, J. (2011). Chaos: Making a new science. Open Road Media.
Grötker, R. (2005). Reine Meinungsmache. German: Technology Review (heise Verlag). http://%5C-www.%5C-heise.%5C-de/%5C-tr/%5C-artikel/%5C-Reine-%5C-Meinungsmache-%5C-277359.%5C-html.
Guala, F. (2002). Models, simulations and experiments. In L. Magnani & N. Nersessian (Eds.), Model-based reasoning: Science, technology, values (pp. 59–74). Kluwer Academic Publishers.
Hammerstein, P. (2003). Why is reciprocity so rare in social animals? A protestant appeal. In P. Hammerstein (Ed.), Genetic and cultural evolution (pp. 83–94). Cambridge, Massachusetts/London, England: MIT Press in cooperation with Dahlem University Press. (Chap. 5).
Harding, S. G. (Ed.). (1976). Can theories be refuted? Essays on the Duhem-Quine thesis. Kluwer.
Harding, A., Keegan, M., & Kelly, S. (2010). Validating a dynamic population microsimulation model: Recent experience in Australia. International Journal of Microsimulation, 3(2), 46–64.
Hartmann, S. (1996). The world as a process: Simulations in the natural and social sciences. In R. Hegselmann et al. (Ed.) Simulation and modelling in the social sciences from the philosophy of science point of view (pp. 77–110).
Heath, B., Hill, R., & Ciarallo, F. (2009). A survey of agent-based modeling practices (January 1998 to July 2008). Journal of Artificial Societies and Social Simulation (JASSS), 12(4), 9.
Hegselmann, R., & Krause, U. (2002). Opinion dynamics and bounded confidence: Models, analysis and simulation. Journal of Artificial Societies and Social Simulation, 5(3), 1.
Imbert, C. (2017). Computer simulations and computational models in science. In L. Magnani, & T. Bertolotti (Eds.), Springer handbook of model-based science (pp. 735–781). https://doi.org/10.1007/978-3-319-30526-4.
Kästner, J., & Arnold, E. (2013). When can a computer simulation act as substitute for an experiment? A case-study from chemisty. In E. Arnold (Ed.), Homepage Eckhart Arnold, preprint.
Kornmesser, S. (2014). Scientific revolutions without paradigm-replacement and the coexistence of competing paradigms: The case of generative grammar and construction grammar. Journal for General Philosophy of Science, 45(1), 91–118. https://doi.org/10.1007/s10838-013-9227-3. ISSN: 1572-8587.
Kuhn, T. S. (1976). Suhrkamp: Die Struktur wissenschaftlicher Revolutionen.
Kuhn, T. S. (1977). The essential tension. Selected studies in scientific tradition and change. The University of Chicago Press.
Lenhard, J. (2007). Computer simulation: The cooperation between experimenting and modeling. Philosophy of Science, 74(2), 176–194. https://doi.org/10.1086/519029.
Lenhard, J., & Winsberg, E. (2010). Holism, entrenchment, and the future of climate model pluralism. Studies in History and Philosophy of Modern Physics, 41, 253–262.
Morgan, M. S. (2003). Experiments without material intervention. Model experiments, virtual experiments, and virtually experiments. In H. Radder (Ed.), The philosophy of scientific experimentation (pp. 216–233). University of Pittsburgh Press.
Morgan, M. S., & Morrison, M. (Eds). (1999). Models as mediators. Perspectives on natural and social science.
Morrison, M. (2009). Models, measurement and computer simulation: The changing face of experimentation. Philosophical Studies, 143, 33–57. https://doi.org/10.1007/s11098-008-9317-y.
Moses, J. W., & Knutsen, T. L. (2012). Ways of knowing. Competing methodologies in social and political research 2nd (first ed. 2007). London: Palgrave Macmillan.
O’Sullivan, D., et al. (2016). Short communication. Strategic directions for agent-based modeling: Avoiding the YAAWN syndrome. Journal of Land Use Science, 11(2), 177–187.
Parker, W. S. (2009). Does matter really matter? Computer simulations, experiments, and materiality. Synthese, 169, 483–496. https://doi.org/10.1007/s11229-008-9434-3.
Peschard, I. (2010). Review of Eric Winsberg’s. Science in the age of computer simulation. University of Chicago Press. Notre Dame Philosophical Reviews (Mar. 31, 2011).
Phelps, S. (2016). An empirical game-theoretic analysis of the dynamics of cooperation in small groups. Journal of Artificial Societies and Social Simulation, 19(2), 4. https://doi.org/10.18564/jasss.3060. ISSN: 1460-7425.
Preston, J. (2016). Paul Feyerabend. In E. N. Zalta (Ed.), The stanford encyclopedia of philosophy. Winter 2016. Metaphysics Research Lab, Stanford University.
Railsback, S. F., Grimm, V. (2012). Agent-based and individual-based modeling. A practical introduction. Princeton University Press.
Rendell, L., et al. (2010). Why copy others? Insights from the social learning strategies tournament. Science, 328, 208–213. https://doi.org/10.1126/science.1184719.
Reutlinger, A., Hangleiter, D., & Hartmann, S. (2017). Understanding (with) toy models. The British Journal for the Philosophy of Science, axx005. https://doi.org/10.1093/bjps/axx005.
Schelling, T. C. (1971). Dynamic models of segregation. The Journal of Mathematical Sociology, 1(2), 143–186. https://doi.org/10.1080/0022250X.1971.9989794. ISSN: 0022-250X.
Schurz, G. (2014). Koexistenz und Komplementarität rivalisierender Paradigmen: Analyse, Diagnose und kulturwissenschaftliches Fallbeispiel. In S. Kornmesser, & G. Schurz (Eds.), Die multiparadigmatische Struktur der Wissenschaften (pp. 47–62). Wiesbaden: Springer Fachmedien Wiesbaden. https://doi.org/10.1007/978-3-658-00672-3_2. ISBN: 978-3-658-00672-3.
Sismondo, S. (2010). An introduction to science and technology studies (2nd ed.). Wiley.
Squazzoni, F., & Casnici, N. (2013). Is social simulation a social science outstation? A bibliometric analysis of the impact of JASSS. Journal of Artificial Societies and Social Simulation, 16(1), 10. ISSN: 1460-7425.
Winsberg, E. (2015). Computer simulations in science. In E. N. Zalta (Ed.), The stanford encyclopedia of philosophy. Summer 2015. Metaphysics Research Lab, Stanford University.
Winsberg, E. (2003). Simulated experiments: Methodology for a virtual world. Philosophy of Science, 70, 105–125.
Winsberg, E. (2009). A tale of two methods. Synthese, 169, 575–592. https://doi.org/10.1007/s11229-008-9437-0.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Arnold, E. (2019). Validation of Computer Simulations from a Kuhnian Perspective. In: Beisbart, C., Saam, N. (eds) Computer Simulation Validation. Simulation Foundations, Methods and Applications. Springer, Cham. https://doi.org/10.1007/978-3-319-70766-2_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-70766-2_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-70765-5
Online ISBN: 978-3-319-70766-2
eBook Packages: Computer ScienceComputer Science (R0)