How Models Fail
A Critical Look at the History of Computer Simulations of the Evolution of Cooperation

Eckhart Arnold
previousupnextTable of Contents
1 Introduction
2 The empirical failure of simulations of the evolution of cooperation
3 Justificatory narratives
4 Bad excuses for bad methods and why they are wrong
    4.1 “Our knowledge is limited, anyway”
    4.2 “One can always learn something from failure”
    4.3 “Models always rely on simplification”
    4.4 “There are no alternatives to modeling”
    4.5 “Modeling promotes a scientific habit of mind”
    4.6 “Division of labor in science exempts theoreticians from empirical work”
    4.7 “Success within the scientific community proves scientific validity”
    4.8 “Natural sciences do it just the same way”
    4.9 Concluding remarks
5 History repeats itself: Comparison with similar criticisms of naturalistic or scientistic approaches
Bibliography

4.5 “Modeling promotes a scientific habit of mind”

Argument: “To me, however, the most important contribution of the modeling enterprise - as distinct from any particular model, or modeling technique - is that it enforces a scientific habit of mind, which I would characterize as one of militant ignorance - an iron commitment to 'I don't know.' That is, all scientific knowledge is uncertain, contingent, subject to revision, and falsifiable in principle. (This, of course, does not mean readily falsified. It means that one can in principle specify observations that, if made, would falsify it). One does not base beliefs on authority, but ultimately on evidence. This, of course, is a very dangerous idea. It levels the playing field, and permits the lowliest peasant to challenge the most exalted ruler - obviously an intolerable risk.” (Epstein 2008, 1.16)

Response: Unfortunately, the modeling tradition discussed in this paper failed completely with respect to all the virtues that Epstein naively believes to be virtues promoted by modeling: It did not readily submit its results to empirical falsification. Where the few and far between attempts of empirical application have been made and failed, the modelers did not learn from failure. (The empirical scientists did learn from the failure by turning away from the RPD-models.) The commitment to “I do not know” becomes a joke if modelers do not dare to come up with concrete empirical explanations or predictions any more. And as far as authority goes, the appeal to “scientific authority” in more or less subtle forms is a common rhetoric device in the modeler's discourse. (See also Moses/Knutsen (2012, 157), Green/Shapiro (1994, 195) and argument 7 below).

Generally, the scientific habit of mind does not at all depend on the use of models. Also, secondary virtues like clarity, explicitness and the like are by no means a prerogative of modelers. Computer simulation studies in particular can become dangerously unclear if the source code is not published or not well structured or not well commented.

page topnext

t g+ f @