How Models Fail
A Critical Look at the History of Computer Simulations of the Evolution of Cooperation

Eckhart Arnold

1 Introduction
2 The empirical failure of simulations of the evolution of cooperation
3 Justificatory narratives
4 Bad excuses for bad methods and why they are wrong
    4.1 “Our knowledge is limited, anyway”
    4.2 “One can always learn something from failure”
    4.3 “Models always rely on simplification”
    4.4 “There are no alternatives to modeling”
    4.5 “Modeling promotes a scientific habit of mind”
    4.6 “Division of labor in science exempts theoreticians from empirical work”
    4.7 “Success within the scientific community proves scientific validity”
    4.8 “Natural sciences do it just the same way”
    4.9 Concluding remarks
5 History repeats itself: Comparison with similar criticisms of naturalistic or scientistic approaches

4.7 “Success within the scientific community proves scientific validity”

Argument: The scientific value of computer simulations in the social sciences cannot be disputed. There is a growing number of research projects, journals, institutes that is dedicated to social simulations. (Variants of this argument are: This book has been quoted so many times, it cannot be all wrong! Or, this article has been published in Science, the authors surely know what they are doing. See also Green/Shapiro (1994, 195), who discuss a similar argument.)

Response: The scientific value of a method, theory, model or simulation is to be judged exclusively on the basis of its scientific merits, i.e. logical reasoning and empirical evidence, and not at all on the basis of its social success. As far as computer simulations are concerned, a survey by Heath et al. (2009) on agent-based simulations revealed that the empirical validation of computer simulations is still badly lacking.

There is one grain of truth in this argument. For those questions, about which one does not know enough to judge the scientific arguments it is best to rely on the judgment of the socially approved specialists. But social success can never be used as an argument within a scientific dispute. After all, it is just the question whether the social success of a theory, model or paradigm was deserved from a scientific point of view.

t g+ f @