How Models Fail
A Critical Look at the History of Computer Simulations of the Evolution of Cooperation

Eckhart Arnold

1 Introduction
2 The empirical failure of simulations of the evolution of cooperation
3 Justificatory narratives
4 Bad excuses for bad methods and why they are wrong
    4.1 “Our knowledge is limited, anyway”
    4.2 “One can always learn something from failure”
    4.3 “Models always rely on simplification”
    4.4 “There are no alternatives to modeling”
    4.5 “Modeling promotes a scientific habit of mind”
    4.6 “Division of labor in science exempts theoreticians from empirical work”
    4.7 “Success within the scientific community proves scientific validity”
    4.8 “Natural sciences do it just the same way”
    4.9 Concluding remarks
5 History repeats itself: Comparison with similar criticisms of naturalistic or scientistic approaches

4.1 “Our knowledge is limited, anyway”

Argument: Our ability to gain knowledge is limited in the social sciences, anyway. Therefore, we have to be content with the kind of computer simulations we can make, even if they are not sufficient to generate empirical explanations.

Response: No one says that we have to use computer simulations in the social sciences. If computer simulations do not work, other methods may still work. As explained earlier, the “Live and Let Live” in World War One cannot really be explained by RPD models, but historiographic methods still work perfectly well in this case.

Even if there exist no alternative methods, we should not accept the existing methods no matter how bad they are. The use of a particular scientific method is justified only, if the results it yields are better than mere speculation and by and large as good as or better than what can be achieved with alternative methods.

Moreover, we should not mistake the failure of a paradigm - say, agent-based simulations or RPD-simulations of cooperation or rational choice theory or sociobiology - for the failure of a science. It is only from the keyhole perspective of the strict adherents to one particular paradigm that the limits of the paradigm appear as the limits of the science or of human cognition as such. In this respect the argument resembles the strategy of silent retreat to false modesty mentioned in the introduction. While it is laudable for a scientist to be modest about one's own claims of knowledge, scientific modesty becomes inappropriate when it gives up any claim of generating empirically falsifiable knowledge.

t g+ f @