How Models Fail
A Critical Look at the History of Computer Simulations of the Evolution of Cooperation

Eckhart Arnold

1 Introduction
2 The empirical failure of simulations of the evolution of cooperation
3 Justificatory narratives
4 Bad excuses for bad methods and why they are wrong
5 History repeats itself: Comparison with similar criticisms of naturalistic or scientistic approaches

5 History repeats itself: Comparison with similar criticisms of naturalistic or scientistic approaches

Although this paper was mostly dedicated to the case of RPD-simulations of the evolution of cooperation, much of the criticism uttered here does not only concern this specific research tradition. In some points it overlaps with like-minded criticism of model oriented or “naturalistic” approaches in the social sciences. In this last part, I'd like to point out some of these overlaps.

In a fundamental, though still constructive criticism Green/Shapiro (1994) have described what they call the “Pathologies of Rational Choice Theory”. The idea that people are by and large rational actors is in itself not necessarily connected to using mathematical models or simulations. But many of the pathologies that Green and Shapiro describe seem to be tied to a particular complex of ontological and methodological convictions lying at the base of the rational choice creed. Among these is a strong commitment to mathematical methods, which are prima facie considered to be more scientific than other methods. What is of interest in this context is what happens when these convictions are frustrated, which they must be, if on the basis of these convictions it is not possible to generate that amount of solid and empirically supported scientific results that had been promised and expected. Will the adherents of the school start to weaken or revise their fundamental convictions? Green/Shapiro (1994, 33ff.) found out that, rather then doing this, adherents of the school applied about any immunization strategy imaginable to protect their theoretical commitment. These strategies ranged form post-hoc theory development over projecting evidence from theory or searching exclusively for confirming evidence to arbitrary domain restrictions. The latter is of particular interest here, because it suggests a historical pattern that is analogous to the one observed in the history of the evolution of cooperation and which I have described as a retreat to false modesty.

According to Green/Shapiro (1994, 45) scientifically legitimate domain restriction is distinguished from arbitrary domain restriction by “specifying the relevant domain in advance by reference to limiting conditions”, rather than ”specifying as the relevant domain: 'wherever the theory seems to work' ”. This problem has - according to their analysis - been particularly acute in the so called ”paradox of voter turnout”, which consists in the fact that people vote at political elections even though the individual influence on the result is so marginal that any cost, even that of leaving the house for voting, should exceed the expected benefit. Now, rational choice theorists have never advanced any convincing explanation for this alleged paradox. Rather, they moved from the question of why people vote to much less ambitious explanations for turnout rate changes (Green/Shapiro 1994, 59). And even here they did not manage to advance more than quite unoriginal hypotheses concerning, for example, the relation between education and the inclination to vote.

In two respects this resembles my results about the scientific tradition of the evolution of cooperation. First of all with regards to the triviality of the results that the simulation-based approach produced in its later stage (like my own “slip stream altruism”-story quoted above). Secondly, with respect to the stepping down from great scientific promises to such humble results. Had Axelrod believed that his simulation models have considerable explanatory power, many of his later followers (e.g. Schüßler) were so careful not to promise too much that one wonders what the simulation method is good for in the context of finding explanations for cooperative behavior, anyway. These coincidences between rational choice theory and RPD-simulations are not surprising, if one assumes that they represent typical immunization strategies of failing paradigms. One difference should be mentioned, though. In the case of rational choice it was largely an empirical failure of the theory, while in the case of the “evolution of cooperation” its was already the failure not to compare the models to empirical research.

Another connection can be pointed out between the criticism launched here and a more recent criticism of the naturalistic paradigm in the political sciences as part of the textbook on competing methodologies in social and political research by Moses/Knutsen (2012, 145-168). Moses and Knutson describe and (modestly) criticize the interconnected complex of ontological and methodological beliefs that makes up the naturalistic paradigm. This complex is composed of elements which are not unlike those that I have discussed as arguments and narratives in the two previous sections. One important element of these is the play with an assumed scientific authority (Moses/Knutsen 2012, 157ff.). Given the many imponderables that surround any theory in the social sciences, including those that profess to employ strictly scientific methods like formal models, Moses and Knutsen come to a similar result as I have: Namely, that this kind of professed scientism is largely a bluff.

t g+ f @