Ecological and Group-Level Studies
Evaluating Epidemiological Research
Learning objectives for this lesson:
- Define ecologic studies, distinguish their exploratory, analytic, and partial-ecologic variants, and explain the rationale for using them despite their central limitation
- List the 3 major categories of variable used in ecologic models and describe their attributes
- Describe the constructs of a linear model at the individual and group levels and constraints on estimating incidence rate ratios at the group level
- Describe the basis of the ecologic and atomistic fallacies
- Describe how within-group misclassification, group-level confounding, and group-level interaction can affect causal inferences
- Identify scenarios where ecologic studies are less likely to produce cross-level inferential errors
- Describe how to integrate individual-level studies with ecologic studies to prevent cross-level inferential errors
- Distinguish ecologic from non-ecologic group-level studies by the level at which inferences are drawn, and appraise published group-level research accordingly
This course was developed by Dr. Kiffer G. Card.
Glossary: Key Terms, People & Concepts
📚 Reference page, available throughout the lesson
This glossary collects the key concepts, people, and ideas you will meet in this lesson. Use it as a reference while you work through the material, or as a review before assessments. Type in the search box to filter entries.
Introduction & Rationale for Group-Level Studies
Introduction and Overview
Earlier lessons worked one unit of analysis: the individual person. Cross-sectional, case-control, and cohort designs all sample people, measure exposures and outcomes on people, and make inferences about people. This lesson changes the unit of analysis to the group (counties, schools, nations) and immediately introduces a problem the previous designs did not have: even when group-level associations are strong and well-measured, you cannot, in general, conclude anything about individuals. The phrase you will spend most of this lesson learning to recognise is the ecological fallacy, demonstrated by Robinson (1950) and named by Selvin (1958). Across the four content sections we move from the rationale for these designs (this section), to the kinds of variables they use (a later section), to the inferential traps they create (a later section), to the analytic strategies that mitigate those traps and the related-but-distinct world of group-level studies that do not commit the fallacy at all (a later section).
Learning Objectives
- 1.1 Define ecologic studies and distinguish exploratory, analytic, and partial-ecologic variants (Morgenstern, 1995).
- 1.2 Explain why the unit of analysis matters and how it shapes the inferences a design can support.
- 1.3 Read group-level examples (arsenic in groundwater, bladder cancer across U.S. states) with the right inferential caveats from the start.
- 1.4 Articulate four legitimate reasons to run an ecologic study (measurement constraint, exposure homogeneity, group-level interest, simplicity) and the cost each carries.
1.1 What Are Ecologic Studies?
Ecologic studies are studies where exposure, outcome, and confounders are all measured at the group level (e.g., townships, counties, nations), but the researcher wants to make inferences about individuals. The groups serve as cluster samples of the population.
Ecologic studies can be exploratory (no direct exposure measurement, looking for associations to guide future research) or analytic (exposure factor is measured and included in the analysis). Some studies are partial ecologic, combining some individual-level variables with group-level variables, which introduces unique inferential challenges.
1.2 Why the Unit of Analysis Matters
Because the group, not the person, is the unit of analysis, the design carries a primary limitation, set out in the box below: the joint distribution of risk factors and disease within groups is unknown, so a group-level pattern and the individual-level truth can point in different directions.
Key Limitation
The primary limitation of ecologic studies is that we do not know the joint distribution of risk factors and disease within groups. This ignorance about within-group associations creates the potential for severe bias when inferring to the individual level.
In plain terms, we know each group's average exposure and its overall disease rate, but not whether it was the exposed members who actually developed the disease. That missing link is what lets a group-level pattern and the individual-level truth point in different directions.
1.3 Examples of Ecologic Studies
Two short examples, arsenic in Idaho groundwater and bladder cancer across US states, make the design concrete before we discuss why anyone would ever use it.
Han et al (2009) examined county-level data on cancer incidence (1991 to 2005) and arsenic levels in groundwater. After adjusting for race, gender, population density, smoking, and body mass index, no significant relationship was found between county-level arsenic exposure and cancer incidence. This illustrates how group-level analyses may fail to detect individual-level associations.
Colli and Kolettis (2010) examined bladder cancer incidence and mortality rates across US states in relation to state-level predictors: smoking prevalence, health insurance coverage, UV index, and water supply type. The ecological analysis identified associations that may or may not reflect individual-level causal mechanisms.
1.4 Rationale for Ecologic Studies
Given the inferential limit just stated, why does anyone run an ecologic study at all? Four reasons recur, each unpacked in the accordion below. The first three explain when ecologic data are the only data you have; the fourth is a warning that simplicity has a real cost. Susser (1994) provides the foundational defence of ecologic analysis as both an outlook and a method in modern epidemiology.
Despite their limitations, ecologic studies are sometimes the only practical approach:
Individual-level measurement of some exposures is impractical or impossible. For example, measuring historical pollution levels or dietary intake for an entire population is expensive. Group-level aggregates (e.g., county-level average pollutant concentration, regional disease prevalence) can serve as proxies.
In some situations, exposure is relatively homogeneous within groups. For instance, all residents of a region receive water from the same supply, all schoolchildren in a district receive the same curriculum-based intervention, or all patients in a clinic receive the same standard of care.
Sometimes the research question is fundamentally about group-level phenomena: Do communities with water fluoridation have lower dental caries rates? Do nations with higher vaccination coverage have lower measles incidence? The group itself is the unit of scientific interest.
Ecologic analysis is often simpler and faster than acquiring and analyzing individual-level data across many groups. However, this simplicity may hide serious methodological problems and inferential errors.
This section has defined ecologic studies, named their central limitation, illustrated the design with two examples, and set out the four reasons it is used despite that limitation. The key takeaways below draw these points together, the reflection and knowledge check that follow close the section, and the next section turns to the kinds of variables these studies use and the model that connects them.
Key Takeaways
- 1.1 Ecologic studies measure exposure, outcome, and confounders at the group level (townships, counties, nations) while the researcher wants to make inferences about individuals. Exploratory studies have no direct exposure measurement and look for associations to guide future research; analytic studies measure the exposure and include it in the analysis. Partial ecologic studies combine some individual-level variables with group-level variables, which introduces its own inferential challenges.
- 1.2 Because the group is the unit of analysis, the joint distribution of risk factors and disease within groups is unknown. We know each group's average exposure and overall disease rate, but not whether the exposed members were the ones who developed the disease, so a group-level pattern and the individual-level truth can point in different directions.
- 1.3 The Idaho groundwater study found no significant county-level association between arsenic and cancer after adjustment, which shows that a group-level null need not mean no individual-level effect. The US bladder cancer study found associations with state-level smoking prevalence, insurance coverage, UV index, and water supply type, which may or may not reflect individual-level causal mechanisms.
- 1.4 Ecologic studies are run when individual exposure cannot practically be measured, when exposure is homogeneous within groups, when the research question concerns group-level effects, and because group-level analysis is simpler and faster. The first three explain when ecologic data are the only data available; the fourth carries a real cost, because simplicity can hide serious methodological problems and inferential errors.
Reflection
Think of a public health issue in your community. How might you design an ecologic study to examine it? What would be your unit of analysis (e.g., neighbourhood, city, province)? What group-level variables would you measure?
Minimum 20 characters required.
1. What distinguishes an ecologic study from other observational study designs?
2. What is a "partial ecologic study"?
3. Which of the following is NOT a rationale for conducting ecologic studies?
Types of Ecologic Variables & The Linear Model
Introduction and Overview
An earlier section motivated the design and named its central liability. To use it carefully, we need a vocabulary for the variables it operates on, a model that connects them, and a clear sense of what that model can and cannot say. This section provides all three.
Learning Objectives
- 2.1 Distinguish aggregate, environmental, and global ecologic variables and explain why the distinction matters for cross-level inference.
- 3.1 Read and interpret the ecologic linear model Yj = β0 + β1X1j + εj and the group-level incidence rate ratio it implies.
- 3.2 Explain why estimating IRG requires extrapolation beyond observed exposure ranges and what that means for inference.
- 3.3 Identify modelling pitfalls: correlation vs regression, standardised outcomes, and cross-level interaction differences.
2.1 Categories of Ecologic Variables
Three major categories of variables can be used in ecologic models, each with different attributes and interpretations. The three flip cards below define each in turn; click each one and notice that they differ in whether the variable has any individual-level analogue at all, a distinction that becomes important when we reach the ecologic fallacy in a later section.
In brief: an aggregate variable summarizes the individuals in a group (the percent vaccinated, the mean BMI, the disease rate); an environmental variable is a physical feature of the place itself (the UV index, a pollutant level, the water supply type); and a global variable has no individual version at all (a law, a policy, or the population density). Aggregate variables summarize individuals and environmental variables stand in for each person's own, usually unmeasured, exposure, while a global variable has no individual counterpart, and that difference is what will matter when we reach the ecologic fallacy.
3.1 The Linear Model in Ecologic Studies
With the variable types in hand, the standard analytic move is a linear regression at the group level. The model in Equation 6.1 is the workhorse; the equations that follow it describe what the regression coefficients mean and where their interpretation gets uncomfortable.
Ecologic studies often use linear regression to model the relationship between group-level exposure and group-level outcome (Equation 6.1):
Where Y is the outcome rate for group j, X1 is the exposure proportion, X2 is a confounder, and ε is the error term. The group-level incidence rate ratio (IRG) is estimated as shown in Equation 6.2:
3.2 Extrapolation and Weighting in Estimating IRG
A major limitation of this approach is that IRG requires extrapolation to groups with 0% and 100% exposure, which may extend far beyond the range of observed data. Additionally, different group sizes may require weighted regression for valid inference.
3.3 Modelling Issues
Several issues arise when modelling ecologic data:
- Correlation vs. regression: Many ecologic studies use correlation coefficients instead of regression coefficients. Regression coefficients estimate the incidence rate difference, which correlation does not provide directly.
- Standardized outcomes: Some studies use standardized mortality ratios (SMRs) rather than crude rates, which may introduce additional complexity.
- Interaction terms: The form of interaction at the group level may differ from the individual level when using linear models at group level and logit models at individual level.
This section has distinguished aggregate, environmental and global variables, set out the ecologic linear model and the group-level rate ratio derived from it, and listed the issues that arise in modelling ecologic data. The key takeaways below draw these points together, the reflection and knowledge check that follow close the section, and the next section turns to the inferential traps these models create, beginning with the ecologic fallacy.
Key Takeaways
- 2.1 Aggregate variables summarize the individuals in a group (the proportion exposed, mean BMI, a disease rate); environmental variables are physical features of the place (UV index, pollutant levels, water supply type); and global variables, such as laws, policies, or population density, have no individual-level analogue at all. Aggregate and environmental variables have an individual-level analogue and only a global variable has none, which is why the distinction matters for cross-level inference.
- 3.1 The ecologic linear model regresses the group outcome rate on the group exposure proportion and other group predictors, with each unit a group rather than a person. The group-level incidence rate ratio, IRG = 1 + β1/β0, compares the rate when everyone is exposed with the baseline rate when no one is, and it is driven entirely by the ratio of the exposure slope to the intercept.
- 3.2 Estimating IRG requires extrapolation to groups with 0% and 100% exposure, which may lie far beyond the range of observed data, so the estimate rests on the assumption that the fitted line holds where there are no observations. Groups of different sizes may also require weighted regression for valid inference.
- 3.3 Many ecologic studies report correlation coefficients, which do not directly estimate the incidence rate difference that a regression coefficient provides. Standardized outcomes such as SMRs add complexity, and the form of interaction at the group level (a linear model) may differ from that at the individual level (a logit model).
Reflection
Consider the three types of ecologic variables. For a study on the relationship between income inequality and mental health outcomes across Canadian provinces, classify each: (a) provincial median income, (b) provincial mental health policy score, (c) average winter temperature.
Minimum 20 characters required.
1. Which type of ecologic variable has NO analogue at the individual level?
2. In an ecologic linear regression model Yj = β0 + β1X1j + εj, what does the group-level incidence rate ratio IRG estimate?
3. Why is using correlation coefficients rather than regression coefficients problematic in ecologic studies?
Inferential Errors & Sources of Ecologic Bias
Introduction and Overview
Earlier sections set up the design and the model. This section is about the trap. It introduces two complementary errors (the ecologic fallacy and the atomistic fallacy) then catalogues the three structural reasons ecologic estimates can mislead. Two interactive simulators let you build the failure modes yourself rather than just reading about them.
Learning Objectives
- 4.1 Define the ecologic fallacy (Robinson, 1950) and explain how a strong group-level association can mislead about individual-level effects.
- 4.2 Define the atomistic fallacy and identify population-level emergent properties (e.g., herd immunity) that have no individual analogue.
- 5.1 Explain how non-differential within-group exposure misclassification biases group-level estimates, and predict the direction of that bias.
- 5.2 Explain how group-level confounding arises, including from factors that are not confounders at the individual level.
- 5.3 Explain how effect modification by group can distort, and even reverse, an ecologic association.
- 6.1 Identify the conditions under which cross-level bias will not occur or is less likely.
- 6.2 Explain how the way zone boundaries are drawn (the modifiable areal unit problem) can change ecologic results.
4.1 The Ecologic Fallacy
The ecologic fallacy is the error of assuming that a group-level association applies to individuals. A finding at the group level (e.g., exposure associated with 3x increased disease risk) does not necessarily mean this is true for individuals. Robinson (1950) gave the classic demonstration, and Selvin (1958) coined the term.
Interactive 6.1 shows the fallacy in a six-scene story of Simpson's paradox, in which a strong positive country-level trend between wine consumption and life expectancy reverses when the view zooms into individuals in France.
Watch a country-level pattern flip when you zoom into individuals. Next ▶ advances scenes.
A 6-scene visualization of Simpson's paradox: wine consumption and life expectancy across countries shows a strong positive trend; zoom into France and the individual-level pattern reverses. Aggregate data answers aggregate questions.
The group-level bias typically exaggerates the association away from the null, but can occasionally reverse the direction of association.
The box below describes the classic example of the fallacy, Durkheim's 1897 observation that regions with higher proportions of Protestant residents had higher suicide rates than predominantly Catholic regions.
In his 1897 study Suicide, Émile Durkheim observed that regions with higher proportions of Protestant residents had higher suicide rates than predominantly Catholic regions. However, this group-level association does not necessarily mean that Protestant individuals were more likely to die by suicide. It is possible that the social environment of predominantly Protestant regions, perhaps characterised by greater individualism or weaker social networks, affected everyone living there, including Catholics. Attributing the group-level finding to individual Protestants would be an ecologic fallacy. Selvin (1958) later used Durkheim's analysis to illustrate this cross-level problem.
The ecologic fallacy arises because within-group variation is hidden when data are aggregated. Individuals who are exposed and individuals who develop disease may be entirely different people within the same group.
The next box describes a task in which the reader builds the fallacy by hand with the simulator in Interactive 6.2, setting a within-group slope and a between-group slope and observing the ecological regression line that an analyst would report.
Hands-on: Ecological Fallacy Explorer
What you'll do: use the simulator below to set a within-group slope (how X relates to Y for individuals inside the same group) and a between-group slope (how group means relate), then watch the ecological regression line that an analyst would actually report. What to take away: when the within-group and between-group slopes have opposite signs, the ecological regression points the wrong way for individuals, and you have built the canonical failure mode by hand. Try the “Robinson 1950” preset first; it reproduces Robinson’s original demonstration.
📊 Interactive 6.2: Ecological Fallacy Explorer
Each colored dot is a person, nested inside a group (e.g., a country or neighborhood). Adjust the within-group slope (how X & Y relate inside a group) and the between-group slope (how group means relate). When the two slopes have opposite signs, the ecological regression lies about the individual reality.
Individual-level data
Each dot = a person, colored by group. Black line = individual-level regression slope.
Group-level (ecological) data
Each large dot = a group's mean. Red line = ecological regression, what an ecological study reports.
The simulator shows that the fallacy is real and structural, the same dataset can support opposite conclusions at the two levels. The same cross-level error can also run in the opposite direction.
4.2 The Atomistic Fallacy
The atomistic fallacy is the opposite error: assuming individual-level findings apply at the group level (Schwartz, 1994; Diez Roux, 1998). Populations have emergent properties not found in individuals. A classic example is herd immunity, a population-level phenomenon with no individual-level counterpart.
The box below sets the two errors side by side, according to the direction in which a finding is wrongly carried from one level to the other.
Key Distinction
The ecologic fallacy occurs when group-level findings are incorrectly applied to individuals. The atomistic fallacy occurs when individual-level findings are incorrectly applied to groups.
Returning to the ecologic fallacy, the next three subsections name the specific mechanisms by which its structural problem becomes a quantitative bias in real ecologic estimates.
5.1 Within-Group Misclassification (Bias)
Three structural sources of ecologic bias are described in this and the next two subsections. Each mechanism corresponds to a different way the within-group slope and the between-group slope can come apart. The first is about exposure measurement, the second is about confounders that look different at the two levels, and the third is about the mathematical mismatch between linear and logit models. The canonical treatment of these mechanisms is Greenland & Robins (1994) and Morgenstern (1995).
Non-differential misclassification at the individual level biases group-level estimates AWAY from the null (opposite direction from individual-level studies). This is given by Equation 6.3:
Where IR is the true individual-level incidence rate ratio, Se is sensitivity, and Sp is specificity. The example of a school CRD study demonstrated how misclassification at the individual level inflates group-level estimates.
Here is the intuition behind that reversal of the usual rule. Mismeasuring exposure squeezes the groups' measured exposure into a narrower range, while their disease rates stay where they are. Fitting a line through the same vertical spread over a shorter horizontal span forces the slope to be steeper, and a steeper slope relative to the intercept is exactly what a larger group-level rate ratio means.
5.2 Group-Level Confounding
Group-level confounding arises from differential distribution of individual-level risk factors across groups. Critically, even factors that are NOT confounders at the individual level can cause confounding at the group level.
Controlling for extraneous risk factors in ecologic analysis generally only removes part of the bias. A worked example of this problem showed confounding that produces biased IRG even when there is no confounding at the individual level.
5.3 Effect Modification (Interaction) by Group
The third source of ecologic bias is effect modification by group, which arises when the association between exposure and outcome differs from one group to another. Because effect modification is always stated on a scale, the box below first reviews the general concept before the text specialises it to modifiers that are properties of groups.
Key Concept: Effect Modification in General
Effect modification is variation in the association between an exposure and an outcome across levels of a third variable, and it is always stated on a scale: additive (for example, comparing risk differences) or multiplicative (for example, comparing risk ratios). An association can be modified on one scale and not on the other. A risk ratio of 2 in two groups whose baseline risks are 1% and 10% gives risk differences of 1 and 10 percentage points, so the effect is modified on the additive scale and not on the multiplicative one.
Statistical interaction is the representation of effect modification by a product term in a model, and the model's scale (linear or logit, for example) fixes the scale on which interaction is assessed. A standard causal diagram does not show effect modification: an arrow states that one variable affects another, not whether the effect is the same for everyone. The group-level case below specialises the idea to modifiers that are properties of groups. HSCI 341 Lesson 7, Section 3 (Controlling Confounding by Design and Stratification) and HSCI 410 Lesson 8, Section 2 (Moderation: When an Association Differs Across Groups) develop it further.
When the rate difference at the individual level varies across groups, non-linearity is introduced: the linear model at group level assumes additivity, but the logit model at individual level is inherently non-linear.
One worked example is striking: effect modification by group completely reversed the direction of association. The true individual-level IR was 5.0, but the ecologic IRG was 0.67, making a harmful exposure appear protective at the group level.
6.1 When Cross-Level Bias Is Less Likely
Cross-level bias does not arise in every ecologic analysis, and its likelihood depends on how exposure and extraneous risk factors are distributed within and across groups. The box below sets out the conditions under which the bias will not occur and those under which it is less likely.
Conditions Minimizing Ecologic Bias
Cross-level (ecologic) bias will NOT occur if:
- The incidence rate difference within groups is uniform across groups, AND
- There is no correlation between group-level exposure and the rate of the outcome in the unexposed
Ecologic bias is LESS likely when:
- There is a large observed range of exposure across groups
- There is small within-group variance of exposure (homogeneous groups)
- Exposure is a strong risk factor varying in prevalence across groups
- Distribution of extraneous risk factors is similar among groups (little group-level confounding)
- Include positive and negative health controls to strengthen ecologic evidence
6.2 The Modifiable Areal Unit Problem
Ecologic results also depend on how the groups themselves are drawn. The box below describes a task with the second simulator, Interactive 6.3, which holds a grid of individual-level data fixed and changes only the zone boundaries drawn around the same people.
Hands-on: MAUP Sandbox
What you'll do: the second simulator below holds the underlying individual-level data fixed and changes only how zone boundaries are drawn around the same people. What to take away: the area-level correlation can change dramatically (even flip sign) without anything about the people changing. This is the modifiable areal unit problem (Fotheringham & Wong, 1991), and it is the reason ecologic results are sensitive to choices that look administrative rather than scientific. Try each zoning preset; same individuals, different ecological “findings.”
🧭 Interactive 6.3: MAUP Sandbox, Same People, Different Zones
A 12×12 grid of people, each with an exposure value (X) and outcome (Y). The individual correlation is fixed; but the area-level correlation depends entirely on how you draw the boundaries. Try each zoning scheme: same people, very different ecological "findings." That is the modifiable areal unit problem (Fotheringham & Wong, 1991).
Individual data (with zoning overlay)
Each tile = a person, shaded by their X value. Black borders show the chosen zones.
Zone-level (ecological) scatter
Each dot = one zone's mean X vs. mean Y. Red line = ecological regression.
This section has defined the ecologic and atomistic fallacies, described three sources of ecologic bias, and set out the conditions under which cross-level bias is less likely, including the sensitivity of ecologic results to how zone boundaries are drawn. The key takeaways below draw these points together, the reflection and knowledge check that follow close the section, and the next section turns to analytic strategies that reduce ecologic bias and to group-level studies that do not commit the fallacy.
Key Takeaways
- 4.1 The ecologic fallacy, demonstrated by Robinson (1950), is the error of assuming that a group-level association applies to individuals. It arises because within-group variation is hidden when data are aggregated: the exposed individuals and those who develop disease may be entirely different people, as in Durkheim's finding on Protestant regions and suicide. Group-level bias usually exaggerates an association away from the null but can reverse its direction, and when the within-group and between-group slopes have opposite signs the ecological regression points the wrong way for individuals.
- 4.2 The atomistic fallacy is the opposite error: assuming that individual-level findings apply at the group level (Schwartz, 1994). Populations have emergent properties not found in individuals, such as herd immunity, a population-level phenomenon with no individual-level counterpart.
- 5.1 Non-differential exposure misclassification at the individual level biases group-level estimates away from the null, the opposite of its usual effect in individual-level studies. Mismeasurement squeezes the groups' measured exposure into a narrower range while their disease rates stay put, so the fitted slope steepens and IRG is inflated.
- 5.2 Group-level confounding arises from the differential distribution of individual-level risk factors across groups, and even factors that are not confounders at the individual level can confound at the group level. Controlling for extraneous risk factors in an ecologic analysis generally removes only part of the bias.
- 5.3 When the individual-level rate difference varies across groups, the additive linear model at the group level no longer matches the non-linear individual-level relationship. In that example a true individual-level IR of 5.0 became an ecologic IRG of 0.67, making a harmful exposure look protective.
- 6.1 Cross-level bias will not occur if the within-group rate difference is uniform across groups and group-level exposure is uncorrelated with the outcome rate in the unexposed. It is less likely with a large range of exposure across groups, homogeneous exposure within groups, a strong risk factor whose prevalence varies across groups, similar distributions of extraneous risk factors, and positive and negative health controls.
- 6.2 Holding the same individuals fixed and redrawing only the zone boundaries can change the area-level correlation dramatically, even flipping its sign. This modifiable areal unit problem (Fotheringham & Wong, 1991) makes ecologic results sensitive to choices that look administrative rather than scientific.
Reflection
A researcher finds that countries with higher per-capita chocolate consumption have more Nobel Prize winners. They conclude that eating chocolate makes individuals smarter. Identify the inferential error being made and explain why this conclusion is problematic. What confounders might explain the group-level association?
Minimum 20 characters required.
1. What is the ecologic fallacy?
2. How does non-differential exposure misclassification at the individual level affect ecologic study estimates?
3. In the lesson's worked example of effect modification by group, the modification caused the ecologic IRG to be 0.67 when the true individual-level IR was 5.0. What does this demonstrate?
Reducing Bias & Non-Ecologic Group Studies
Introduction and Overview
An earlier section catalogued the failure modes. This section is the constructive response. The first half names analytic strategies that can pull part of the individual-level signal out of group-level data; the second half makes a distinction the textbook treatment often glosses over, that some studies use group-level data without committing the ecologic fallacy at all, because their inferences also stay at the group level.
Learning Objectives
- 7.1 Describe analytic strategies for reducing ecologic bias, including multilevel modelling (Diez Roux, 1998) and the Wakefield & Haneuse two-phase design (2008).
- 7.2 Describe how two-phase sampling and the linkage of area-level exposures to individual records bring individual-level data to bear on group-level questions.
- 8.1 Distinguish ecologic from non-ecologic group-level studies based on the level at which inferences are drawn, not the level at which data are collected.
- 8.2 Apply Rose’s distinction between “the etiology of a case” and “the etiology of incidence” to research questions you encounter.
- 8.3 Use the Dufault & Klar (2011) reporting-quality findings as a reading checklist for any ecologic study you appraise.
7.1 Analysing Ecologic Data to Minimize Bias
Ecologic bias is less of a problem when certain conditions are met (see Section 3, 6.1 When Cross-Level Bias Is Less Likely). Additionally, researchers can employ specific analytical strategies:
Each of the three strategies that follow brings further information to bear on group-level data: individual-level data in multilevel modelling and in the two-phase design, and prior knowledge about within-area probabilities and contextual effects in the third.
- Multilevel modelling (MLM): Combines individual-level and group-level data to distinguish individual-level effects from contextual (group-level) effects (Diez Roux, 1998). Validates assumptions and investigates random effects.
- Two-phase design (Wakefield & Haneuse, 2008): Links individual-level data with ecologic data using outcome-dependent sampling within groups, reducing the need for complete individual-level information.
- Prior information: Importance of prior knowledge about within-area probabilities and contextual effects when making inferences.
7.2 Linking Individual-Level and Area-Level Data
The two-phase logic also applies to individual-level studies: the Case-Control Studies lesson describes two-stage (two-phase) sampling, in which inexpensive data are collected on a large group and more detailed, usually more expensive, data are collected on a strategically selected subsample. A second route away from purely ecologic analysis is to attach area-level exposures to individual records. The Cohort Studies lesson describes CANUE (the Canadian Urban Environmental Health Research Consortium), whose standardised postal-code-level estimates of air pollution, greenness, and neighbourhood deprivation can be linked to individual health records, so that individual outcomes can be analysed with multilevel models that include both individual-level and area-level variables.
The strategies above try to repair ecologic data so it can speak about individuals. The next subsection makes the alternative move: keep the inference at the group level and acknowledge that groups are sometimes the right unit of scientific interest in their own right.
8.1 Non-Ecologic Group-Level Studies
Not all studies using group-level data are ecologic studies. The box below sets out a critical distinction.
The Key Difference
When variables are measured at the group level AND inferences remain at the group level → NOT ecologic. The group as the aggregate-scale of interest studying how group-level characteristics (population density, policies, social environments) affect group-level outcomes.
Examples of non-ecologic group-level studies include:
- Health promotion programs targeting communities, with outcomes measured at the community level
- Vaccination campaigns evaluated by population-level coverage and population-level incidence
- Organizational interventions in clinics or hospitals, with organization-level outcomes
8.2 The Question of Inference Level
Rose (2001) distinguished two key epidemiological questions:
- "What is the etiology of a case?" This is an individual-level question, seeking to understand why a particular person became ill.
- "What is the etiology of incidence?" This is a population-level question, seeking to understand why populations have different disease rates.
Both questions are important; the appropriate level of analysis depends on the research question. The atomistic fallacy arises when researchers reduce all phenomena to individual-level explanations, ignoring emergent group properties.
8.3 Quality of Current Ecologic Research
Dufault & Klar (2011) reviewed the reporting quality of 125 cross-sectional ecologic studies published since 2000 in six major epidemiology journals and found concerning patterns:
- 58% omitted important information about the choice of study design, so only about 42% explained it
- 49% did not discuss whether their research was susceptible to the ecological fallacy
- 31% omitted important information about the ecologic nature of their study
- 64% of the investigators who adjusted their outcomes for age or sex did so improperly
- Simple linear regression was the most common analytic approach (31% of studies)
This section has described strategies for reducing ecologic bias, distinguished ecologic studies from group-level studies whose inferences remain at the group level, and reviewed the reporting quality of published ecologic research. The key takeaways below draw these points together, and the reflection and knowledge check that follow close the section and the lesson.
Key Takeaways
- 7.1 Multilevel modelling combines individual-level and group-level data to separate individual-level effects from contextual effects and to investigate random effects (Diez Roux, 1998). The Wakefield & Haneuse two-phase design (2008) links individual-level data to ecologic data through outcome-dependent sampling within groups, reducing the need for complete individual-level information, and prior knowledge about within-area probabilities and contextual effects also informs inference.
- 7.2 Two-stage sampling collects inexpensive data on a large group and more detailed data on a strategically selected subsample. Linking area-level exposures, such as CANUE's postal-code-level estimates of air pollution, greenness, and neighbourhood deprivation, to individual health records lets individual outcomes be analysed with multilevel models that include both individual-level and area-level variables.
- 8.1 When variables are measured at the group level and inferences also remain at the group level, a study is not ecologic and the cross-level fallacies do not apply. Community health promotion programs, vaccination campaigns evaluated by population coverage and incidence, and organizational interventions with organization-level outcomes are examples.
- 8.2 Rose (2001) distinguished the etiology of a case, an individual-level question, from the etiology of incidence, a population-level question. Both are important, the appropriate level of analysis depends on the research question, and reducing all phenomena to individual-level explanations commits the atomistic fallacy.
- 8.3 Dufault & Klar (2011) reviewed 125 ecologic studies and found that 58% omitted important information about the choice of an ecologic design, 49% did not discuss whether their research was susceptible to the ecological fallacy, and 64% of those adjusting for age or sex did so improperly. These gaps make a useful checklist for appraising any ecologic study.
Reflection
Consider a city that wants to evaluate whether its new bicycle-sharing program has reduced cardiovascular disease rates. Would an ecologic design or individual-level design be more appropriate? What are the trade-offs? How might you combine both approaches using multilevel modelling?
Minimum 20 characters required.
1. Which of the following conditions makes ecologic bias LESS likely?
2. What is multilevel modelling (MLM) in the context of ecologic studies?
3. When is a group-level study NOT considered an ecologic study?
Final Assessment
Bringing It All Together
This lesson moved from the rationale for ecologic designs through the variables and models they use, to the structural traps that make them treacherous, and finally to the analytic and conceptual responses to those traps. The arc was deliberate: every step was preparation for being able to read a published group-level study without overclaiming or underclaiming what its data actually support.
The single most important idea to carry forward is the one Robinson demonstrated in 1950: a finding that holds at the group level need not hold at the individual level, and the reverse (the atomistic fallacy) is equally damaging in the other direction. The three structural sources of ecologic bias (within-group misclassification, group-level confounding, effect modification by group) explain why the cross-level move can fail. Multilevel modelling and the two-phase design explain how to bring some of that signal back without abandoning the group-level data we already have. Rose’s distinction between the etiology of a case and the etiology of incidence reminds us that the right unit of analysis depends on the question being asked, not on which is more familiar from earlier lessons.
A later lesson takes the next step: how the constructs we measure (exposure, outcome, confounder) are defined and operationalised in the first place, the conceptualisation step that determines whether any of the cross-level inference machinery in this lesson can do the work we want it to.
Key Takeaways from this lesson
- Ecologic studies measure exposure and outcome at the group level while seeking inferences about individuals; they may be exploratory or analytic, and partial ecologic studies mix individual- and group-level variables, which introduces its own inferential challenges. Because the joint distribution of exposure and disease within groups is unknown, they are justified mainly when individual measurement is impractical, exposure is homogeneous within groups, or the question is about groups.
- Aggregate variables summarize individuals, environmental variables describe the place, and global variables have no individual-level analogue; aggregate and environmental variables have an individual-level analogue, and only global variables have none.
- The ecologic linear model estimates a group-level rate ratio that requires extrapolation to 0% and 100% exposure, well beyond most observed data, and modelling choices such as correlation in place of regression, standardized outcomes, and the scale of interaction add further pitfalls.
- The ecologic fallacy (Robinson, 1950) is the error of assuming a group-level association applies to individuals; the atomistic fallacy (Schwartz, 1994) is the symmetric error in the other direction.
- Three structural sources of ecologic bias, within-group exposure misclassification, group-level confounding, and effect modification by group, explain why the cross-level move so often fails, and each can distort an association: misclassification tends to inflate it away from the null (and with a strong effect can give an impossible value), while group-level confounding and effect modification can exaggerate or reverse it.
- Cross-level bias is less likely when exposure varies widely across groups but little within them and extraneous risk factors are similarly distributed, yet ecologic results remain sensitive to how zone boundaries are drawn.
- Multilevel modelling and the Wakefield & Haneuse two-phase design are the principal analytic strategies for recovering individual-level signal from group-level data, and linking area-level exposures to individual records extends the same idea.
- Not every group-level study is ecologic: when both data and inferences stay at the group level, the cross-level fallacies do not apply, Rose’s “etiology of incidence” is a legitimate and important question in its own right, and reporting gaps identified by Dufault & Klar give a checklist for appraising published work.
Reflection
Reflecting on this lesson, describe a scenario from public health or your field of interest where an ecologic study design would be the most practical and informative approach. What safeguards would you implement to minimize the risk of the ecologic fallacy?
Minimum 20 characters required.
Final Knowledge Assessment
1. Ecologic studies differ from other observational designs primarily because:
2. A "partial ecologic study" is one where:
3. Which is an example of a global variable?
4. Aggregate variables in ecologic studies are:
5. In the ecologic linear model, the group-level incidence rate ratio IRG requires:
6. The ecologic fallacy refers to:
7. The atomistic fallacy is:
8. Non-differential exposure misclassification in ecologic studies biases estimates:
9. Group-level confounding in ecologic studies:
10. In the lesson's worked example of effect modification by group, the modification caused:
11. Ecologic bias is LESS likely when:
12. Multilevel modelling (MLM) helps address ecologic bias by:
13. A study measuring the effect of a city's water fluoridation policy on community-level dental health (with inferences remaining at the community level) is:
14. According to Dufault and Klar (2011), what proportion of ecologic studies adequately justified the choice of ecologic design?
15. Rose (2001) distinguished between two key epidemiological questions. Which pair correctly represents them?