How Models Fail
A Critical Look at the History of Computer Simulations of the Evolution of Cooperation

Eckhart Arnold

1 Introduction
2 The empirical failure of simulations of the evolution of cooperation
3 Justificatory narratives
4 Bad excuses for bad methods and why they are wrong
    4.1 “Our knowledge is limited, anyway”
    4.2 “One can always learn something from failure”
    4.3 “Models always rely on simplification”
    4.4 “There are no alternatives to modeling”
    4.5 “Modeling promotes a scientific habit of mind”
    4.6 “Division of labor in science exempts theoreticians from empirical work”
    4.7 “Success within the scientific community proves scientific validity”
    4.8 “Natural sciences do it just the same way”
    4.9 Concluding remarks
5 History repeats itself: Comparison with similar criticisms of naturalistic or scientistic approaches
Bibliography

4.3 “Models always rely on simplification”

Argument: Models, by their very definition, rely on simplifications of reality. If a model wouldn't simplify it would be useless as a model. After all, the best map of a landscape would be the landscape itself, but then it would be useless as a map. (A typical example is Zollman (2009) who relies on this argument in his criticism of mine. See also Green/Shapiro (1994, 191) who discuss a similar argument in the context of rational choice theory.)

Response: On the other hand it is obvious that there must be some limit to how strongly a model may simplify reality. For otherwise any model could be a model for anything. So, where is the borderline between legitimate simplification and illegitimate oversimplification? A possible answer could be that a model is not oversimplified as long as it captures with sufficient precision all causally relevant factors of the modeled phenomenon with respect to a specific research question, i.e. all factors that are liable to determine the outcome of this question. In all other cases we should be very careful to trust an explanation based on that model alone.

At this point two replies are common: 1) That no one claims such an explanatory power for his or her own models. But then, what is the point of modeling, if models do not help us to explain anything? 2) That the research question did not require that all causally relevant factors have to be captured by one and the same model. However, if a model concentrates only on some causal factors, then these must at least be discernible empirically from other factors at work. Unfortunately, this is often not possible and certainly not with most RPD models. (See also Arnold (2014b, 367f.).)

As far as RPD-simulations are concerned it appears clear to me that these are far too simplified to be acceptable representations of reality. One could object that they help us to understand the mechanism of reciprocal altruism as such. This is already one step back from claiming that RPD-models are an effective means for investigating the evolution of cooperation, because now it is merely claimed that they are illustrating a mechanism. However, for this purpose a single model would be sufficient. One does not need dozens of them. Plus, how and why reciprocal altruism works in principle has perfectly well been conceptualized by Robert Trivers (1971) many years earlier with a single simple equation.

t g+ f @