The Dark Side of the Force: When computer simulations lead us astray and "model think" narrows our imagination. (revised version, October 2006)

Eckhart Arnold

1 Introduction
2 Different aims of computer simulations in science
3 Criteria for explanatory simulations
4 Examples of Failure: Axelrod style simulations of the “evolution of cooperation”
    4.1 Typical features of Axelrod style simulations
    4.2 How Axelrod style simulations work
    4.3 The explanatory irrelevance of Axelrod style simulations in social sciences
    4.4 Do Axelrod style simulations do any better in biology?
5 Conclusion
Bibliography

4.4 Do Axelrod style simulations do any better in biology?

The sceptical conclusion about Axelrod-style simulations the last section closes with becomes even more pronounced when we look at examples from biology, a field where the obstacles against formal modeling are much smaller than in social sciences. Not being a biologist myself, it would of course be difficult for me to estimate the usefulness of Axelrod style simulations for the explanation of cooperative behavior in biology. Luckily, there exists a comprehensive survey by the biologist Lee Allen Dugatkin on “Cooperation among Animals” (Dugatkin 1997) that pays some particular attention to the manifold of game theoretical models and computer simulations that have come up in the aftermath of Axelrod's “Evolution of Cooperation”. In the beginning of his book Dugatkin lists various game theoretic computer simulations and their results (Dugatkin 1997, p. 24ff.), which - being the results of computer simulations alone - are purely theoretical of course. The major part of his book consists of a survey of the empirical research on the various instances of cooperative behavior that can be found in the animal kingdom. Interestingly, there exists not a single instance of cooperative behavior in the animal kingdom to which any of these computer simulations could be applied in a strict sense.

This is not to say that biologists did not try to do so. The attempt has been made, for example, to apply Robert Axelrod's and William D. Hamilton's theory of the evolution of cooperation to the behavior of predator inspection that is found among various types of shoal fishes. In an early paper by Manfred Milinski on the topic (Milinski 1987), Milinski tries to find out - with the help of an inventive experimental setup - whether pairs of inspecting fishes play Tit for Tat like Axelrod and Hamilton had postulated it for the repeated prisoner's dilemma. In order to do so Milinski also assesses (or rather estimates) the payoff parameters of Axelrod's model as applied to this particular case. Like Axelrod in the case of the “live and let live” system in the trench war of the First World War, he confines himself to an assessment of the ordinal relations between the payoff parameters, which, as we have seen, is unfortunately not sufficient, since the simulation is sensitive to the cardinal values of the payoff parameters. In later studies on the topic of predator inspection the attempt to explain this type of behavior with Axelrod's theory of the “evolution of cooperation” seems to have been completely dropped. In a paper that appeared ten years after the first study, Milinski and Parker, even leave the question open, whether pairwise predator inspection is an instance of cooperative behavior at all (Milinski/Parker 1997).[7] A major methodological problem is that - despite some very ingenious experiments - it is extremely difficult to measure or to estimate reliably both the risk a fish runs when inspecting a predator and the fitness relevant payoff a fish receives from inspecting. (The former has to some degree been achieved by Milinski and Parker, but the latter remains an open riddle).

As Dugatkin summarizes the situation in the concluding chapter of his book, there exists, with one exception, no case of cooperative animal behavior where the payoff parameters required as input for the game theoretical computer models could be measured. Therefore, it is no surprise that none of the many Axelrod style simulations of the evolution of cooperation could be applied strictly to any of the empirical instances of cooperation in biology. And it is very doubtful whether this type of simulations (which remains remote from concrete empirical research and rests purely on “plausible” assumptions) is of any use for biologists at all. Another leading exponent of the game theoretic approach in biology puts it the following way: “Why is there such a discrepancy between theory and facts? A look at the best known examples of reciprocity shows that simple models of repeated games do not properly reflect the natural circumstances under which evolution takes place. Most repeated animal interactions do not even correspond to repeated games.” (Hammerstein 2003, p. 83) And after a long discussion of problems that the study of cooperative behavior of animals faces the same expert concludes: “Most certainly, if we invested the same amount of energy in the resolution of all problems raised in this discourse, as we do in publishing of toy models with limited applicability, we would be further along in our understanding of cooperation.” (Hammerstein 2003, p. 92)

One might object that maybe some of the models can be further developed so that they actually fit some of the empirical examples of reciprocity. And there would indeed be some truth in this objection: It does not matter whether one starts constructing a model with a certain empirical application case in mind and builds it around measurable quantities (bottom up approach) or whether one starts with arbitrary plausible assumptions and only later on tries to adjust the model to specific empirical situations (top down approach). But one way or the other our models and the empirical processes they are meant to explain should be brought together. For, just because we have a model that shows us that for this or that reason cooperation evolves or breaks down, we cannot conclude in any empirical case of the evolution or breakdown of cooperation that it did so by virtue of the very same causes for which it did in the model. It could also have been the effect of quite different causes. Unless there is a “close fit” between model and reality we will never know.

The downside of computer simulations that do not achieve a “close fit” between model and reality is not only that they do not work, but that they tend to give us a wrong picture of the subject matter at hand. The “skew towards reciprocity in theoretical literature” on altruism (Dugatkin 1997, p. 167f.) is most probably also due to the fact that the simulation business lost contact to the empirical research in this field. Instead of seeking to achieve a “close fit” between model and reality, the tradition of Axelrod-style modeling of the “evolution of cooperation” largely proceeded a different course. Computer simulation followed after computer simulation, each of them changing the basic configuration in some way or other or trying the addition of new and different parameters. But most of these simulations never got to the ground of empirical testability. This way, however, computer simulations only lead us away from the real scientific problems.

[7] For a summary of the heated debate that took place about the Tit For Tat strategy in biology in the meantime see Dugatkin's book on animal cooperation (Dugatkin 1997, p. 67-70).

t g+ f @