Validation of Computer Simulations from a Kuhnian Perspective

von Eckhart Arnold

1 Introduction
2 Kuhn's philosophy of science
3 A revolution, but not a Kuhnian revolution: Computer simulations in science
4 Validation of Simulations from a Kuhnian perspective
5 Summary and Conclusions

1 Introduction

Thomas Kuhn (1976) famously introduced the term paradigm to characterize the set of background beliefs and attitudes shared by all scientists of a particular discipline. According to Kuhn these beliefs and attitudes are mostly centered around exemplars of good scientific practice as presented in the textbook literature, but classical texts, specific methodological convictions or even ontological commitments can also become important for defining a paradigm. Furthermore, paradigms comprise shared convictions as well as unspoken assumptions of the group of researchers (Kuhn 1976, postscript). An important function of paradigms is that they both define and limit what counts as relevant question and legitimate problem within a scientific discipline.

Kuhn's concept of a paradigm is closely connected with his view of how science develops. According to Kuhn phases of normal science where science progresses within the confinements of a ruling paradigm are followed by scientific revolutions which, in a process of creative destruction, lead to a paradigm-shift. Scientific revolutions are triggered by the accumulation of problems that are unsolvable within the ruling paradigm (so called anomalies). With an increasing number of anomalies scientists grow unsatisfied with the current paradigm and start to look for alternatives - a state of affairs that Kuhn (1976, ch. 7/8) describes as the crisis of the ruling paradigm. Then, a paradigm-shift can occur that consists in a thoroughgoing conceptual reorganization of a scientific discipline or, as the case may be, the genesis of a new sub-discipline. Unless there is a crisis, the search for alternative paradigms is usually suppressed by the scientific community.

This theory could be relevant for computer simulations and their validation. Because computer simulations are sometimes characterized as a revolutionary new tool that blurs the distinction between model and experiment, the question can be asked if this tool brings about or requires new paradigms of validation. Under validation I understand a process which allows to test whether the results of a scientific procedure adequately capture that part of reality which they are meant to explain or to enable us to understand. It is widely accepted that for theories or theoretical models, the process of validation consists in the empirical testing of their consequences by experiment or observation, which in this context is also often described as verification or falsification or, more generally, as confirmation.[1] The question then is, if the same still holds for computer simulations, that is, if computer simulations also require some form of empirical validation before they can be assumed to inform us about reality.

For the purpose of this paper, I understand empirical validation in a somewhat wider sense that does not require strict falsification, but merely any form of matching theoretical assumptions with empirical findings. In this sense, a historian checking an interpretation against the historical sources can also be said to validate that interpretation. However, I assume that proper validation always includes an empirical component and I therefore use the terms “validation” and “empirical validation” interchangeably in the following.

In the following, I first summarize Kuhn's philosophy of science (Sec.~ 2). Then I list some of the dramatic changes that computer simulations have brought about in science and - in order to forestall possible misunderstandings - explain why these changes are not scientific revolutions in the sense of Kuhn (Sec.~ 3). In the main part of this chapter (Sec.~ 4), I then examine the validation of simulations from a Kuhnian perspective. Relating to the discussion about the relation between computer simulations and experiments I argue that computer simulations can clearly be distinguished from real experiments and, therefore, do not require a new paradigm of validation. In principle, validating simulations is just like validating theory. I continue by examining whether computer simulations aggravate the problem of theory choice that is associated with the so called “Duhem-Quine-thesis” (Harding 1976), which I deny. Finally, I examine some of the issues that the validation of social simulations and in particular agent-based-models raises from the point of view of Kuhn's philosophy of science. For the lack of commonly accepted standards of validation, it seems unclear whether this field has already reached a state of “normal science” with established paradigms of validation. Because the practices of validation vary greatly in this field, a general conclusion is not possible, however. I therefore confine myself to discussing the issue with respect to selected examples.

[1] In the realm of computer simulations the term verification is, somewhat confusingly, reserved for checking wether the simulation software is free from programming errors (so called “bugs”) and whether it is faithful to the mathematical model or theory on which it is based. The term validation is used for the empirical testing of the simulation's results. See also Chapter 4 (Murray/Smith 2019) in this volume.

t g+ f @