Tools or Toys?
On Specific Challenges for Modeling and the Epistemology of Models in the Social Sciences

Table of Contents

7.2 Consequences for philosophers of science

The object of the philosophy of models and simulations is to reconstruct how models and simulations contribute to the generation of scientific knowledge. In order to do so the philosophy of models asks what models are and, more importantly, how models prove. The latter question is more important, because it helps us to draw the line between proper use and improper use of models. In this respect the philosophy of models takes up a similar task as the philosophy of science in general does with the demarcation problem. To draw a line between proper use and improper use of models is a particularly important task for the philosophy of simulations in the social sciences, because, as a matter of fact, the scientific value of many social simulations appears to be rather doubtful (Arnold 2008, Hammerstein 2003).

>From what has been said previously, it should be clear that the epistemic situation for models and simulations in the social sciences is somewhat different from that in the natural sciences and engineering, although the transition is of course smooth and interlocking. Paying proper attention to this difference leads, as I believe, to the following conclusions for the the philosophy of models and simulations:

1. Models “mediate” differently in the social sciences. While in the natural sciences “models as mediators” are linked to exact and well-confirmed background theories on the theory-side and to precisely measurable data on the empirical side, the standard case in the social science appears to be quite different.

   On the theory-side there are no precise, well-confirmed and content-rich background theories. Often modelers help themselves with plausible assumptions to feed their models (Hegselmann/Krause 2002), which unfortunately adds quite a bit of arbitrariness to the models right at the beginning. In other cases the assumptions are result of a careful empirical assessment of the target system (Siebers et al. 2010).

   On the empirical side, the data that the models are related to are only sometimes precisely measurable quantities. Often the “data” consists of or is embedded in narrative descriptions of situations which need to be strongly stylized before they can be fed into models.

   Summing it up: The “mediation” concept of models is of comparatively more limited applicability in the social sciences, because i) there are no theories on the one end of the “mediators” and ii) there is more involved in the process of mediation than merely models or a cascade of models.

    
2. The analogy between simulations and experiments is harder to justify. In an experiment we learn something about nature from nature. A simulation in contrast “generates new knowledge on the basis of existing knowledge”[12] . This is not to say that we cannot learn something about nature from simulations. We can do so if the existing knowledge already is fairly comprehensive and well-assessed. In the natural sciences, part of the existing knowledge consists of powerful and empirically well-confirmed theories. Under this condition we can learn something about nature from “computer experiments” that apply these theories to particular research questions (as in the ribosome example) or that put more specialized theories to the test.

   Because of the lack of well confirmed and powerful (i.e. structure-rich) background theories, simulations in the social sciences usually do not have this quasi-experimental status. The may attain such a status if the assumptions that are built into the simulation are - even without a background theory - at least empirically well-assessed for the simulated scenario. But frequently this is not the case and if it is not the case then the “experimentation”-terminology used in connection with mere computer simulations (as in (Hegselmann/Flache 1998, 3.11) for example) can be misleading. For, what these simulations show are only the consequences of more or less arbitrary assumptions, but not the behaviour of the simulated entities in nature.

   Philosophers of science should be aware that there is a categorial distinction between experiments and simulations. The analogy between experiments and simulations works only under certain favorable conditions such as the existence of comprehensive and reliable background knowledge. These conditions are usually not met in the social sciences.

    
3. Validation and research designs for models and simulations differ in the social sciences. As far as validation ist concerned, the lack of empirically well confirmed background theories means that the assumptions entering into the model need to be assessed individually for the scenarios to which the model is to be applied. (The not uncommon practice to rely on merely “plausible assumptions” is rather unsatisfactory and should not be sanctioned by a critical philosophy of science.) Also, because the model input (e.g. assumptions, measured parameter values, tried and trusted modeling practices) is typically less reliable in the natural sciences, direct empirical validation of the simulation results becomes more important.

   It stands to reason that as a consequence of these differences the kinds of research design that are most successful in the social sciences are different form those in the natural sciences. If we follow Alexandrova's examination of the use of auction models (Alexandrova 2008) then a successful research design is one where models function as “open formulae” for generating causal hypotheses in a trial and error approach that includes models as well experiments as complementary elements of the research process. At the same time other accounts of modeling which are somewhat more in line with the research logic in the natural sciences fail to adequately capture the showcase of the auction design (Alexandrova 2008, p.\ 387-393).

   While similar trial and error research can also occur in the natural sciences, it might turn out that it is the standard case of a successful simulation-research design in the social sciences. This suffices to give the activity of modeling or simulating a distinct flavor in the social sciences.

    
4. Philosophers of science should refrain from rationalising bad practices. Philosophy of science is not so much a descriptive but a critical enterprise. Its aim is to reconstruct how science generates reliable knowledge about the world. But the philosophy of science should also criticise scientific practise when it is flawed. In this respect the aim of the philosophy is not only to understand how science generates knowledge but also to critically examine whether it actually does.

   If, as in the case of Robert Sugden, one has reason to wonder that “authors typically say very little about how their models relate to the real world” (Sugden 2009, p.\ 25) then the most salient explanation is that these models are simply not fit to teach us anything about the world (Cartwright 2009, p.\ 48ff.). Philosophers should allow for this possibility and refrain from rationalising bad methodological practice.

   This is the more important, because some of the common research designs for simulations in the social sciences appear to be heavily flawed. For example, the (implicit) research design that lies at the basis of many simulations of “the evolution of cooperation” in the tradition initiated by Robert Axelrod (Axelrod 1984) which works by constructing purely theoretical simulations and then drawing generalizing conclusions from the results is flawed, because the generalizing conclusions are not sufficiently warranted (Binmore 1998, p.\ 313-319). And a more modest variant of this research design, which consists in constructing purely theoretical simulations and then never drawing any empirical conclusions at all, is also not convincing, because it raises the question why we should be interested in models from which we cannot learn anything about the world.

   As in science and philosophy rational argument ought to decide about the truth and falsehood of opinions and not the number of supporters, philosophers of science need not to be impressed by how widespread certain faulty research designs are.

But why is it important to be aware of these epistemological differences of simulations in the social sciences and simulations in the natural sciences? The answer is that our explicit or implicit epistemological ideas have a regulatory function when designing research programs. Wrong epistemological ideas can entail the long-term failure of research program. If we believe that the epistemological conditions in the social sciences are just the same as in the natural sciences then we will expect simulation studies that are designed on the role model of the natural sciences to sooner or later yield good results. Any failure to do so will in the first place be considered as a failure of the particular simulation study and not of the research program. If we are aware of the differences between social sciences and natural sciences, then we might still consider it worth while to learn and apply techniques for social simulations that have been sucessful in the natural sciences. But in cases where these fail we will much sooner consider the possibility that the research design was inappropriate and that the research program needs to be readjusted.

[12] This is the very clear expression used by Jen Schellinck and Richard Webster in their talk at the Models and Simulations 4 Conference in Toronto, May 2010. To avoid any kind of misunderstanding one might add that “simulations generate knowledge exclusively on the basis of existing knowledge”.

t g+ f @