Tools or Toys?
On Specific Challenges for Modeling and the Epistemology of Models in the Social Sciences

Eckhart Arnold

1 Introduction
2 The role of models in science
3 Why computer simulations are merely models and not experiments
4 The epistemology of simulations at work: How simulations are used to study chemical reactions in the ribosome
5 How do models explain in the social sciences?
6 Common obstacles for modeling in the social sciences
    6.1 Lack of universal background theories
    6.2 Pluralism of Paradigms
    6.3 Why parsimony is a vice and not a virtue
    6.4 “Wholistic” nature of many phenomena in the social sciences
    6.5 Difficulties of measurement
    6.6 Pluralism of scientific styles
7 Conclusions

6.5 Difficulties of measurement

The empirical data in the social sciences does often not have the form of measurable quantities and where it does, it is often difficult to measure it precisely.

An example for non-quantitative data would be historical sources like international treaties. Examples for quantitative data that are well defined and can be measured precisely are money or the number of inhabitants of a country at a given time. Quantitative magnitudes that are less well defined and hard to measure would be the power a state has in relation to other states or the utility a consumer derives from the consummation of a certain good. To make this point a little clearer the latter examples shall be discussed in slightly more detail.

While power seems to be a magnitude that has an order of greater or smaller, any comparison remains almost inevitably vague. This is especially true when different forms of power like economic power and military power are to be compared. Thus, despite of what Bertrand Russell had hoped some time ago (Russell 1938, p.\ 10), it is impossible to form a concept of power that works similar to that of energy in physics, where different forms of energy, say potential energy and heat energy, can be measured and compared precisely.

As regards utility: In order to apply utility theory in the same way as, say, the concept of force in physics, one would have to determine peoples' preferences and measure the cardinal utility values they attach to them with a reasonable degree of accuracy. Even in purely economic contexts this is often well-neigh impossible. Money, to be sure, can be measured, but then what prompts peoples' actions is not money but the utility they derive from money or other means.

Now all this is of course well known, but what is easily overlooked are the restrictive consequences these facts have for the range of reasonable modeling in the social sciences. Assume, for example, a scientist wants to explain why the victorious powers of the Second World War were willing to agree to the unification of Germany in 1990 and she wants to do so by using a game theoretical model of the negotiation process. Now, the raw data available consists of protocols (if available) of the two plus four negotiations, communiques and news releases of the involved parties, treaty drafts, the final treaty and the like. Before any game theoretical model can be fed with these data they would need to be transformed into quantitative parameters through a careful process of interpretation. Given that there is always a certain range of reasonable interpretation the interpreted data must be considered quite noisy. The need of interpretation also occurs on the way back when interpreting the results of a formal model so that they make sense in terms of the phenomena that the model is about.

One might of course deny that this is a suitable problem for the application of a game theoretical model. But if this is denied then already one important lessen is learned: Due to the nature of the data occurring in the social sciences, formal modeling is sometimes not a reasonable option. And if it is not denied

then it does at least highlight some of the specific challenges that the application of mathematical models faces in the social sciences due to difficulties of measuring data quantitatively.

The epistemological consequences can be summarized as follows:

  1. Establishing a link between a model and empirical reality can be difficult as it may require careful interpretation of empirical facts. Other than in the natural sciences the last step in the chain of models leading from theory to empirical reality may not simply be a model of data or phenomena but a hermeneutical interpretation of data. (By “hemerneutical” I mean “involving the interpretation and understanding of a product of human cognition by a human agent”.)
  2. Because of the difficulties of quantitative measurement, great strain is placed on the robustness of models. In order to draw valid conclusions, a model must be robust with respect to variations of the values of its input parameters within the range of measurement inaccuracies. The larger the measurement inaccuracies or - in cases where hermeneutical interpretation takes the place of measurement - the range of acceptable interpretations, the more robust, therefore, the model must be.

t g+ f @