How Models Fail
A Critical Look at the History of Computer Simulations of the Evolution of Cooperation

Eckhart Arnold

1 Introduction
2 The empirical failure of simulations of the evolution of cooperation
3 Justificatory narratives
4 Bad excuses for bad methods and why they are wrong
    4.1 “Our knowledge is limited, anyway”
    4.2 “One can always learn something from failure”
    4.3 “Models always rely on simplification”
    4.4 “There are no alternatives to modeling”
    4.5 “Modeling promotes a scientific habit of mind”
    4.6 “Division of labor in science exempts theoreticians from empirical work”
    4.7 “Success within the scientific community proves scientific validity”
    4.8 “Natural sciences do it just the same way”
    4.9 Concluding remarks
5 History repeats itself: Comparison with similar criticisms of naturalistic or scientistic approaches
Bibliography

4.6 “Division of labor in science exempts theoreticians from empirical work”

Argument: There exists division of labor in science. Model builders are not responsible for the empirical application of their models, but they are mere suppliers. If the empirical scientists fail to test or otherwise make use of models, it is not the modelers that should be blamed.

Response: But modelers need to take into account the conditions and restrictions that empirical research imposes, otherwise they run the danger of producing models that can never, not even under the most favorable circumstances, be applied empirically. In the case of the Axelrod-tradition it is clearly the modelers that must take the blame, because they failed to learn from the failures of early attempts at empirical application like Milinski's (1987). And they never worried about the restrictions under which empirical work struggles in the potential application fields of their models.

Now, one might say that this is also true for much of mathematics, and still mathematics has often proven to be applicable, even in cases that no one had guessed before. But surely it is not a good research strategy to rely on later to come historical coincidences of science. Plus, there is an important difference between mathematics and models. Mathematics deals with general structures, while simulation-models like the RPD represent particular example cases (comparable to a concrete calculation in mathematics). From a technical point of view most models in the Axelrod tradition remain fairly trivial, while mathematics could - if worst comes to worst - still be justified by its high intellectual level which allows to ascribe an innate value to it.

t g+ f @