When Falsification Fails

Abstract

This study investigated the effectiveness of a falsification logic at early and late stages of the hypothesis testing process. The subject's task was to discover the "laws of motion" in a computerized Artificial Universe. Twenty science students, after detailed instructions on hypothesis testing, worked independently on the task in the presence of the experimenter. Subjects who were instructed to generate multiple hypotheses and follow a falsification logic did significantly worse than those receiving no such instructions. The manipulation of the early and late knowledge stages had no effect, apparently because few subjects attended to the information intended to put them into an advanced state of knowledge. A combination of quantitative and qualitative analyses suggested the following generalizations. 1. While falsifiability may be a useful criterion for demarcating science from non-science, the deliberate attempt to use falsification may be counterproductive in complex environments. 2. The use of a falsification logic may simply be beyond the cognitive capacity of people in complex environments; it is very difficult to try to prove something false that one believes true. 3. The use of metaphors and analogies, largely precluded from functionality in most laboratory reasoning tasks, appears spontaneous and useful in richer environments. 4. In complex environments, it is evident that a richer classification scheme for the types of hypothesis tests than currently accepted may be required.