A nice property of blogs is that they provide an opportunity to share ideas that, for one reason or another, never made it to publication. Here is a letter to Science that I wrote a few years ago which was bounced. It is in response to a piece called “Testing hypotheses: prediction or prejudice” by Peter Lipton (Science 307: 219-221; see replies by several others in Science 308: 1409-1412).

Flaws of the fudge factor

Lipton’s (1) analysis of common explanations for the intuitive notion that prediction is more convincing than accommodation is interesting and apt. His own contribution to the discussion – namely that differences in the opportunity for “fudging” in support of a favoured hypothesis drive this disparity – is less compelling. A few things to consider in this regard:

1) Who can fudge? Lipton (1) argues that “fudging” is more problematic in accommodation than in prediction, but this appears to be based on the assumption that the predictions are made by one scientist and tested independently by another. Certainly, this pertains to some fields of physics, but in a great many instances in the life sciences, the predictor and tester are one and the same. Thus, there is ample opportunity for (unintentional) “fudging” of predictive tests in support of a preferred hypothesis at all stages, from experimental design to data collection to interpretation. Moreover, one could make the argument that “tests” based on accommodation, which use data generated independently by many other authors and in which errors are random with respect to the hypothesis being tested, are more – not less – objective. It also should be self-evident that if an author ignores contradictory data during an accommodation-based test, then other members of the scientific community are likely to expose this omission.

2) A question of scale. If the comparison being made is in regard to a single datum, then indeed prediction may be far more convincing than accommodation. However, if the scales differ, as they often do in real life, such that the prediction relates to a tractable and therefore simple test but the accommodation deals with multiple, independent types of information, then the latter may be considered much more convincing. Therefore, it matters a great deal that prediction often tests one aspect of an issue whereas accommodation can include a much broader “consilience of inductions” (2).

3) Prediction as confirmation. The history of science is replete with examples in which predictive testing, while undoubtedly particularly convincing, has served mainly to confirm ideas that had been developed primarily on the basis of accommodation. This has included some of the most revolutionary breakthroughs in science, including relativity, natural selection, atomic theory, and genetics (both Mendelian and molecular). The question of which is the superior way to advance science is a false dichotomy.

4) An alternative hypothesis: x + y > x. Perhaps the most important point overlooked by Lipton (1) is that hypotheses that are not compatible with existing data – i.e., which have not already achieved accommodation – are rejected before ever being tested by prediction. In this sense, a more parsimonious explanation for the perception that prediction is more powerful than accommodation is that the former is a second-order test that only occurs after the first has been completed. In other words, prediction seems more powerful because its very existence implies that a more fundamental criterion, accommodation, has already been met. It is a mere truism that two tests will be more convincing than one, especially if the second test indicates inherently that the first has been passed.

References
1. Lipton, P. Testing hypotheses: predictions and prejudice. Science 307: 219-221, 2005.

2. Whewell, W. The Philosophy of the Inductive Sciences, Founded Upon Their History, 1840.