Tools or Toys?
On Specific Challenges for Modeling and the Epistemology of Models in the Social Sciences

Eckhart Arnold

1 Introduction
2 The role of models in science
3 Why computer simulations are merely models and not experiments
4 The epistemology of simulations at work: How simulations are used to study chemical reactions in the ribosome
5 How do models explain in the social sciences?
6 Common obstacles for modeling in the social sciences
7 Conclusions

5 How do models explain in the social sciences?

It is much harder to see how models or simulations contribute to the understanding of phenomena in the social sciences than in the natural sciences, because in many cases social science models are highly stylized. Often the degree of idealization is so strong that the models are obviously unrealistic and it becomes hard to see how they represent their target system at all (see Hammerstein (2003) for an example of this problem). The textbook literature on economics, which is the social science in which the use of models is the most pervasive, defends the use of strongly simplified models with the standard argument that because reality is much too complex to be described directly, we would not be able to gain any understanding at all without strongly simplified models (Mankiw 2004, p.\ 22ff.). As it is at the same time obvious that not any arbitrary simplifications are permissible, the question inevitably arises what kind of simplifications are permissible and what kind of simplifications are not. Unfortunately, the economic textbook literature offers no satisfactory answer to this question. To rely on the predictive success of otherwise unrealistic simulations offers no solution, because the predicitive success of economic models is notoriously weak (Betz 2006). And if this is true for economics then the situation for models in other social sciences like sociology or political sciences must be expected to be even worse.

Not surprisingly, therefore, the ongoing debate about the proper use and the epistemic value of models in the social sciences is characterized by a wide diversity of different and often contradictory viewpoints. To an outsider it could almost appear as if mathematical models and computer simulations are used in the social sciences without a common understanding of what they are good for. Among the views taken in this debate are the following:

  1. Models as predictive devices: As just mentioned, Friedman defends models as tools for generating empirical predictions (Friedman 1953). Unfortunately, due to the usually poor predictive success only very few social science models can seriously be defended on this ground. This is especially the case if the predictive success of elaborated models is no better than that of simple naive prediction methods (Betz 2006, ch. 2/3).

    The basic rationale of this concept of models can be described as this: Models help us to understand the world by generating successful predictions. Other than that they do not need to be particularly realistic.

  2. Models as experiments: Some authors strongly emphasize the analogies between models and experiments (Morrison 2009, Maeki 2005). In their view most models and most experiments share more or less the same features (e.g. more or less close resemblance to a target system, controlled environment, potentially unpredictable and surprising results) and it is therefore often more a matter of convenience and feasibility which of the two is to be preferred under which circumstances. But, as has been argued earlier, the analogy quickly breaks down if we consider experiments that test the empirical truth or falsehood of fundamental theories (experimentum crucis). Most of the authors advocating this view are, of course, aware that it does not work for all types of models and experiments. But even if this is taken into account, there is the constant danger of forgetting about the epistemic primacy of the empirical side of science. (After all it is the empirical world that our theories must be adjusted to and not the other other way round.)

    The bottom-line of the “models as experiments” conception is: Models and Simulations help us to understand the world much like experiments do, e.g. by representing a real-world target system in a controlled environment that allows us to test assumptions.

  3. Models as isolating devices: The view that models are “isolating tools” has some likeness to the “models are experiments” view, because when conducting experiments one often tries to isolate the causal relation under study as good as possible from all disturbing influences. To the adherents of this view it does not matter, how the isolated system is arrived at; if by shielding from other influences (experiments) or by including just as much as is needed to model the causal relation in the first place (models) (Maeki 2009). The value of this analogy is disputed by others, however (Kourikoski/Lethinen 2009, p.\ 127). And with regards to the epistemic situation the same caveats as for the analogy between models and experiments hold.

    If models are understood as isolating tools then the important question is, if and how we can learn something from isolating models about the real world. As Nancy Cartwright argues, this depends on the scientific field. According to her, in comparison with physics very little can be learned from isolating models in economics. And one can probably safely generalize this finding to all social sciences. The possible explanation for this limitation that compared to the natural laws in physics economics has only few general principles (Cartwright 2009) links well with our previous contention that strong background theories provide a good ground for successful modeling.

    Models help us to understand the world, because they allow us to study the functioning of causal mechanisms in isolation which in the real world are usually mixed up with other mechanisms.

  4. Models as credible counterfactual worlds: Robert Sugden's “credible world” account of models (Sugden 2000, Sugden 2009) is motivated by the observation that models in the social sciences often do not represent any particular target systems. Rather surprisingly, “authors typically say very little about how their models relate to the real world” (Sugden 2009, p.\ 25). Determined to find a rationale behind this modeling practice nonetheless, Sugden develops his “credible worlds” account. According to Sugden models are neither isolations or abstractions nor do they merely serve the purpose of “conceptual exploration”. But they constitute “counterfactual credible worlds”. Because the models (in economics) are by their very nature “counterfactual” it would be misplaced to demand that they be realistic. Yet, they need to be “credible” in order to allow us to draw inductive conclusions from them.

    This account of models raises more questions than it answers: In what sense can a world that is “counterfactual” still be credible? And what are the criteria by which the credibility of a “counterfactual” model must be judged?

    Regardless of how these questions might be answered, the role of models according to this view can be characterized as follows: Models do not represent particular target systems but they constitute paralell counterfactual worlds. By being “credible” they allow us to draw inductive conclusions about the real world.

  5. Models as incredible counterfactual worlds: In contrast to Sugden, Kourikoski/Lethinen (2009) do not assume that the counterfactual worlds of models need to be “credible” to be of good service to our understanding of the real world. Quite the contrary, a model may very well contain counterfactual or even incredible assumptions. Varying counterfactual assumptions plays a crucial role in what Kourikoski and Lethinen call “derivational robustness analysis”. This is a procedure by which the robustness of a model's “substantive” assumptions can be tested by varying its “auxiliary” assumptions. If the the substantive assumptions still produce the same result no matter what varying counterfactual auxiliary assumptions are made in the model, they can be considered robust and we are entitled to draw the inductive conclusion that even if the counterfactual auxiliary assumptions would be replaced by realistic assumptions, the same results can be expected.[9]

    Though it does raise questions, this is an interesting robustness concept that certainly deserves further exploration. If it proves to be sound than it offers a strategy how a model can be hardened - up to a certain limit, if the substantive assumptions are not to be tautologies - by a pure model to model comparison.

    The bottom line is: If a model is counterfactual or even incredible it can still help us to understand the world.

  6. Models as partial explanations: Drawing on the conceptual framework of C.G.Hempel, Aydinonat describes models as partial explanations (Aydinonat 2007). This means that models capture one possible cause of a phenomenon that can have several causes which may differ from instance to instance. Aydinonant gives this account in the course of a case study on Schelling's neighborhood segregation model, a model that explains the macroeffect of segregated neighborhoods with the micromotive of individuals having a weak preference against living in a neighborhood that is dominated by an ethnic group other than their own. Saying that this model provides a partial explanation means that particular instances of neighborhood segregation may have been caused by the factor that the model describes but could also have been caused by other factors. For, neighborhood segregation can also result from housing prices in connection with a difference in average income levels of different ethnic groups. An empirical assessment is needed to decide which causes were effective in a particular instance of the phenomenon.

    Although it is probably not appropriate for all types of models, Aydinonat's account is a very convincing one: It provides a clear and convincing idea of how and why models may contribute to explanations in the social sciences. And it can almost immediately be transformed into a research design.

    Summarized, this account of models says: Models are partial explanations that describe possible causes of phenomena of a specific type. For providing a full explanation of a particular phenomenon, the model must be sufficiently robust and it must be chacked empirically against other possible causes of the phenomenon.

  7. Models as “open formulae” An even more defensive reading of the role of models in social sciences that has been suggested by Alexandrova (2008) who treats them as open formulae. By this it is meant that models are merely templates or schema to generate causal hypotheses about the world. If models are only templates for causal claims then they do not carry any direct burden of epistemological justification any more, but the burden lies on the hypotheses that are produced from the template-models.

    Just as the previous one this account has the merit of suggesting a research design where the use of models interacts with that of experiments. The kind of iterative research that results is described by Alexandrova with respect to the example of auction design (Alexandrova 2008, p.\ 384ff.).

    Summary: Models serve as templates for the generation of hypotheses.

  8. Models as tools for conceptual exploration: Finally, and at the other end of the spectrum models and simulations can be regarded as a purely theoretical device that serves the purpose of conceptual exploration. There is no doubt that models and simulations can be used to explore the implications of our theories and concepts. The question is whether they are good for anything else. What is assumed here is that, unless they are empirically applied to particular target systems, models and simulations do not serve any other purpose than that of conceptual exploration.

    Bottom line: Models serve primarily theoretical functions like that of exploring the implications of concepts and theories.

The diversitiy of views on simulations, exemplified by the above list, is not simply a consequence of the fact that models and simulations are used for different purposes in the social sciences. For, some of the contradictory views like the “credible worlds” (Sugden 2009), “incredible worlds” (Kourikoski/Lethinen 2009), “isolation” (Cartwright 2009, Maeki 2009) and “partial explanation” account (Aydinonat 2007) have been proposed by their authors in the same discussion and under consideration of exactly the same examples.

The situation is somewhat embarrassing because as Sugden has observed “authors typically say very little about how their models relate to the real world” (Sugden 2009, p.\ 25). And it is not because the answer is so obvious that they remain silent. Otherwise, why would there be such a debate? One obvious approach to resolving the debate would be to go through the accounts that have been advanced one by one and either find an account that is the most adequate or to arrive at some kind of synthesis. But, apart from the fact that this procedure would be rather tedious, it is also not guaranteed that it leads to the desired result. For, it is unclear which criteria a good account of models ought to fulfill. Therefore, before entering into any discussion about epistemological accounts of models it might be advisable to ask for the reasons why the research logic of models and simulations in the social sciences is not at all obvious.

So, how is it possible that scientists use models, yet nobody seems really able to tell “how their models relate to the real world” (Sugden 2009, p.\ 25)?

[9] This procedure is somewhat similar to the “de-idealization” procedure proposed by Ernan McMullin 25 years earlier (Mcmullin 1985, Alexandrova 2008). Only that when “de-idealizing”, the auxiliary assumptions must gradually be replaced by more realistic assumptions. One can conjecture that when both alternatives are available “de-idealization” is the safer method. Derivational robustness analysis could then be understood as a kind of second-best alternative to “de-idealization” when the latter is not available.

t g+ f @