What's wrong with social simulations? |
| Table of Contents |
In this paper I will try to answer the question: Why is the epistemic value of so many social simulations questionable? Under social simulations I understand computer simulations of human interaction as it is studied in the social sciences. The reason why I consider the epistemic value of many social simulations as questionable is that many simulation studies cannot give an answer to the most salient question that any scientific study should be ready to answer: “How do we know it’s true?” or, if specifically directed to simulation studies: “How do we know that the simulation simulates the phenomenon correctly that it simulates?” Answering this question requires some kind of empirical validation of the simulation. The requirement of empirical validation is in line with the widely accepted notion that science is demarcated from non-science by its empirical testability or falsifiability. Many simulation studies, however, do not offer any suggestion how they could possibly be validated empirically.
A frequent reply by simulation scientists is that no simulation of empirical phenomena was intended, but that the simulation only serves a “theoretical” purpose. Then, however, another equally salient question should be answered: “Why should we care about the results?” It is my strong impression that many social simulation studies cannot answer either this or the first question. This is not to say that the use of computer programs for answering purely theoretical questions is generally or necessarily devoid of value. The computer assisted proofs of the four color theorem (Wilson 2002) are an important counterexample. But in the social sciences it is hard to find similarly useful examples of the use of computers for purely theoretical purposes. In any case, the social sciences are empirical sciences. Therefore, social simulations should contribute either directly or indirectly to our understanding of social phenomena in the empirical world.
There exist many different types of simulations but I will restrict myself to agent-based and game theoretical simulations. I do not make a sharp difference between models and simulations. For the purpose of this paper I identify computer simulations just with programmed models. Most of my criticism of the practice of these simulation types can probably be generalized to other types of simulations or models in the social sciences and maybe also to some instances of the simulation practice in the natural sciences. It would lead too far afield to examine these connections here, but it should be easy to determine in other cases whether the particulars of bad simulation practice against which my criticism is directed are present or not.
In order to bring my point home, I rely on the survey by Heath et al. (2009) on agent-based modeling practice for a general overview and on two example cases that I examine in detail. I start by discussing the survey which reveals that in an important sub-field of social simulations, namely, agent based simulations, empirical validation is commonly lacking. After that I first discuss Thomas Schelling’s well-known neighborhood segregation model. This is a model that I do not consider as being devoid of epistemic value. For, unlike most social simulations, it can be empirically falsified. The discussion of the particular features that make this model scientifically valuable will help us to understand why the simulation models discussed in the following fail to be so.
The simulation models that I discuss in the following are simulations in the tradition of Robert Axelrod’s “Evolution of Cooperation” (Axelrod 1984). Although the modeling tradition initiated by Axelrod has delivered hardly any tenable and empirically applicable results, it still continues to thrive today. By some, Axelrod’s approach is still taken as a role model (Rendell et al. 2010a, 208-209), although there has been severe criticism by others (Arnold 2008, Binmore 1994, Binmore 1998).
Finally, the question remains why scientists continue to produce such an abundance of simulation studies that fail to be empirically applicable. Leaving possible sociological explanations like the momentum of scientific traditions, the cohesion of peer groups, the necessity of justifying the investment in acquiring particular skills (e.g. math and programming) aside, I confine myself to the ideological background of simulation scientists. In my opinion the failure to produce useful results has a lot to do with the positivist attitude prevailing in this field of the social sciences. This attitude includes the dogmatic belief in the superiority of the methods of natural sciences like physics in any area of science. Therefore, despite frequent failure, many scientists continue to believe that formal modeling is just the right method for the social sciences. The attitude is well described in Shapiro (2005). Such attitudes are less often expressed explicitly in the scientific papers. Rather they form a background of shared convictions that, if not simply taken for granted as “unspoken assumptions”, find their expression in informal texts, conversations, blogs, keynote speeches. I discuss Joshua Epstein’s keynote lecture “Why Model?” (Epstein 2008) as an example.
|