In 19th century Europe, suicide rates were higher in countries that were more heavily Protestant, the inference being that suicide was promoted by the social conditions of Protestantism (Durkheim, 1897).  This is an “ecological inference”—a conclusion about individual behavior drawn from data about aggregate behavior.

The Protestant countries, of course, were different from the Catholic countries in many ways besides religion (the problem of “confounding”). Moreover, Durkheim’s data did not tie individual suicides to any particular religious faith: he had rates for countries, not for religious denominations within geographic areas.

The problem of confounding must be dealt with in any observational study.  But the second problem is specific to ecological studies: putative causes and effects are measured for groups rather than individuals.  Moreover, the analytic interest is in one kind of grouping– for Durkheim, religion– whereas data are available for an entirely different sort of grouping (geography).

If there is no confounding, the expected difference between effects for groups and for individuals is “aggregation bias”; in general, the difference is partly attributable to confounding and partly to aggregation bias. The “ecological fallacy” consists in thinking that relationships observed for groups necessarily hold for individuals: if countries with more Protestants tend to have higher suicide rates, then Protestants must be more likely to commit suicide.

Ecological studies can provide useful clues, but conclusions about individuals are in general only weakly supported by data on groups. The source of the problem is confounding and aggregation bias.  Indeed, as shown by Robinson (1950), individual-level relationships can easily be reversed by aggregation.  This is “Simpson’s Paradox” for the correlation coefficient.

Statistical procedures have been proposed for disentangling individual-level from group-level behavior, including “ecological regression” and “cross-level” or “hierarchical” regression models. However, each method makes its own rather strong behavioral assumptions, which seem implausible when stated explicitly.  For instance, ecological regression makes the “constancy assumption.” According to this assumption, with an application like Durkheim’s, individual behavior cannot depend on geographical location. Protestants all over Europe must have similar propensities to commit suicide; and Catholics are homogeneous too.

Generally, the identifying assumptions in the models cannot be validated by the data.  Moreover, in test situations where individual-level data are available– so that estimates can be compared to reality– the track record of the models is mixed at best. For additional discussion from various perspectives, see the references below.