# Lesson 6: Ecological and Group-Level Studies

*Companion-podcast transcript • Sarah & Kiffer*  
*Office Hours episode to listen to after working through the lesson*

---

**Sarah:** Welcome back to Office Hours. I'm Sarah.

**Kiffer:** And I'm Kiffer. This episode goes with Lesson six, Ecological and Group-Level Studies.

**Sarah:** It's meant for after you've worked through the lesson. We'll bring in some perspective and some critique, spend time on the questions students tend to find thorny when the data describe groups, and work through some extra examples, including a few that go further than the ones on the lesson page.

**Kiffer:** Along the way there are a few places where we ask you to work something out before we give the answer. When that happens, you'll hear a few seconds of quiet. Pause the audio if you'd like more time, and come back when you have an answer, even a rough one.

**Sarah:** We have four questions. First, what goes missing when data are grouped, and why can a strong pattern across groups mislead us about people? Second, how do you read the group-level rate ratio, and what does extrapolation do to it? Third, why does measurement error push a group-level estimate away from the null? And fourth, when is a study that uses group data not ecologic at all, and is the ecological fallacy sometimes invoked too quickly?

**Kiffer:** Question one. The key limitation from the lesson is that we don't know the joint distribution of exposure and disease within groups. That phrase is easy to repeat and harder to picture, so let's build it with made-up numbers. Imagine two hypothetical towns, each with one thousand adults. In Town A, twenty percent of adults have some exposure, and there are thirty cases of a disease. In Town B, sixty percent are exposed, and there are fifty cases.

**Sarah:** So at the group level, the town with three times the exposure has more disease. If those two towns were points on a graph, the line would slope upward, and the obvious reading is that the exposure is harmful.

**Kiffer:** That's the reading most people would make. So here's the question. Using only those totals, is it possible that inside both towns, the exposed people actually have the lower risk? Take a few seconds.

*(Pause)*

**Sarah:** Let me try it. Town A has two hundred exposed people and eight hundred unexposed. If only two of the thirty cases are among the exposed, that's two out of two hundred, or one percent, and the other twenty-eight cases among eight hundred unexposed people is three and a half percent. Town B has six hundred exposed and four hundred unexposed. If twenty-four cases are among the exposed, that's four percent, and the remaining twenty-six among four hundred unexposed is six and a half percent. So yes. In both towns the exposed do better, and the totals still come out at thirty and fifty.

**Kiffer:** Exactly. And the same totals fit the opposite story just as well. If the risk were two percent in everyone unexposed and seven percent in everyone exposed, Town A would have sixteen cases plus fourteen, which is thirty, and Town B would have eight plus forty-two, which is fifty. The group data can't tell those two worlds apart.

**Sarah:** What made my version possible was that the unexposed people in Town B had a much higher risk than the unexposed people in Town A, six and a half percent compared with three and a half. Something about Town B raised everyone's risk.

**Kiffer:** That's the right thing to notice. In the lesson's list of conditions, cross-level bias can't occur when the rate difference is the same in every group and the rate among the unexposed doesn't move with the group's exposure level. You broke the second condition. The town with more exposure also had a higher background rate, and that background difference did all the work.

**Sarah:** Is there a real example where the reversal actually happened?

**Kiffer:** Robinson's 1950 paper is the classic one. He used the 1930 United States census. Among individuals, being born outside the country was weakly associated with being unable to read, with a correlation of about positive 0.12. Across states, the correlation between the share of residents born abroad and the illiteracy rate was about negative 0.53, and a later reanalysis put it at about negative 0.46. The explanation is that immigrants tended to settle in states where the native-born population had higher literacy.

**Sarah:** So the states with more immigrants looked more literate, even though immigrants themselves were somewhat less likely to read. The background literacy of each state did the work.

**Kiffer:** Right, and it's why the lesson treats the missing joint distribution as the central limitation. Every other problem in this lesson is a particular way that missing information shows up.

**Sarah:** Question two is about the ecologic linear model. Kiffer, can you set up a made-up example and let me try it?

**Kiffer:** Sure. Suppose twenty hypothetical regions report their disease rate per one hundred thousand person-years and the proportion of their residents who are exposed. The fitted line says the rate equals twenty plus thirty times the proportion exposed. In these regions, the proportion exposed runs from thirty percent to sixty percent. What is the group-level incidence rate ratio? Take a few seconds.

*(Pause)*

**Sarah:** Okay. I'll compare the most exposed region with the least exposed. At sixty percent exposure, the rate is twenty plus thirty times 0.6, which is thirty-eight. At thirty percent, it's twenty plus thirty times 0.3, which is twenty-nine. Thirty-eight divided by twenty-nine is about 1.31. So the rate ratio is about 1.3.

**Kiffer:** Let's do a sanity check on that. In the two regions you compared, who is exposed and who is unexposed? Think about what the rate ratio is supposed to compare.

**Sarah:** Both regions are mixtures. One is sixty percent exposed and the other is thirty percent exposed, so I compared two blends of exposed and unexposed people. The group-level rate ratio compares a region where everyone is exposed with one where no one is. So I need the line at a proportion of one and at a proportion of zero.

**Kiffer:** Right. Try it that way, reading the line at zero and at one.

**Sarah:** At zero, the rate is the intercept, twenty. At one, it's twenty plus thirty, which is fifty. Fifty divided by twenty is 2.5. And that matches the formula from the lesson, one plus the slope divided by the intercept, which is one plus thirty divided by twenty, or 2.5.

**Kiffer:** That's it. And the slope itself, thirty per one hundred thousand person-years, is the group-level rate difference. Now here's the uncomfortable part. Did any region in the data have zero exposure, or full exposure?

**Sarah:** No. The data only run from thirty to sixty percent. So both the twenty and the fifty are predictions for groups that don't exist in the data.

**Kiffer:** That's the extrapolation problem. The 2.5 assumes the straight line holds all the way out to zero and all the way out to one. If the true relationship curves outside the observed range, the rate ratio can be badly off, and nothing in the data would warn you.

**Sarah:** And a correlation coefficient alone would give us no slope and no intercept, so no rate difference or rate ratio.

**Kiffer:** That's why the lesson prefers regression coefficients to correlations. I'd add one practical point. If some regions have five thousand residents and others have five hundred thousand, their rates aren't equally precise, so the regression usually needs to be weighted by group size.

**Sarah:** Question three is the one I find hardest to hold onto. In individual-level studies, non-differential misclassification of a yes-or-no exposure usually pulls an estimate toward the null. In ecologic studies, it pushes the estimate away from the null, and I want to see why.

**Kiffer:** Let's build that with numbers, because it's the harder example in this episode. Suppose the true group-level line is a rate of twenty plus forty times the true proportion exposed, per one hundred thousand person-years. So the true rate ratio is one plus forty divided by twenty, which is three. Now suppose exposure is measured in each person with a sensitivity of ninety percent and a specificity of ninety percent, and the errors are the same in every region. Which way will the estimated rate ratio move from three, and roughly how far? Take a few seconds.

*(Pause)*

**Sarah:** My instinct from earlier lessons is that it should shrink toward one. But let me actually work it out before I trust that. A region where nobody is exposed will still have ten percent of people wrongly classified as exposed, because the specificity is ninety percent. So it looks ten percent exposed.

**Kiffer:** Good. Now apply the same reasoning to a region where everyone is truly exposed.

**Sarah:** Ninety percent of them are correctly classified, so it looks ninety percent exposed. But the disease rates don't change. The truly unexposed region still has a rate of twenty, and the truly exposed region still has a rate of sixty, which is twenty plus forty.

**Kiffer:** So now fit the line through those two measured points, ten percent exposure with a rate of twenty, and ninety percent exposure with a rate of sixty.

**Sarah:** The rate goes up by forty while the measured exposure goes up by 0.8, so the slope is forty divided by 0.8, which is fifty. Going back from ten percent down to zero, I subtract fifty times 0.1, which is five, so the intercept is fifteen. The rate ratio is one plus fifty divided by fifteen, which is about 4.33. So it moved away from the null, from three up to about 4.3, and my instinct was wrong.

**Kiffer:** And you can check it with the lesson's misclassification formula. The true rate ratio minus one is two. The denominator is the sensitivity plus the specificity times the true rate ratio, minus the true rate ratio, so 0.9 plus 2.7 minus three, which is 0.6. Two divided by 0.6 is about 3.33, and one plus that is about 4.33. Same answer.

**Sarah:** So the intuition is that the errors squeeze the measured exposures together, from a span of zero to one down to a span of 0.1 to 0.9, while the rates stay put. The same rise in rate over a shorter run makes a steeper line.

**Kiffer:** That's it, and the steeper slope also lowers the intercept, so the ratio of slope to intercept grows from both ends. There's a warning sign worth knowing here too. If the true effect is large and the classification is poor, the fitted intercept can drop to zero or below. A predicted rate below zero is impossible, and the rate ratio that comes out is meaningless. When you see an implausibly huge or negative rate ratio from an ecologic regression, suspect the measurement.

**Sarah:** And this is only one of the three sources of ecologic bias in the lesson. Group-level confounding and effect modification by group can stack on top of it, in either direction.

**Kiffer:** Which is the general lesson of question three. In ecologic studies, the rules of thumb from individual-level designs about which way a bias runs don't carry over, so you have to reason it through for the specific design.

**Sarah:** Question four. When is a study that uses group-level data not ecologic? It depends on the level of the inference. If the conclusions stay about groups, the cross-level fallacies don't apply. Let me give everyone three made-up studies to classify. In the first, provinces with a higher share of adults who walk or cycle to work have lower diabetes rates, and the authors conclude that a person who starts cycling to work will lower their own risk. In the second, the same provincial data are used to conclude that provinces that raise active commuting can expect lower provincial diabetes incidence.

**Kiffer:** And here's the third made-up study. Researchers survey ten thousand individual adults, attach a greenness score to each person from their postal code, and model each person's diabetes using both their own characteristics and their neighbourhood's score. How would you classify each of the three? Take a few seconds.

*(Pause)*

**Sarah:** The first is an ecologic study making an individual-level claim, so it's exposed to the ecological fallacy. The provincial data say nothing about whether the people who cycle are the people who avoid diabetes. The second uses identical data and keeps the conclusion at the provincial level, so it's a non-ecologic group-level study.

**Kiffer:** I agree with both of those. The third study measures diabetes in each person and links an area-level exposure to their individual record, then uses a multilevel model with individual and neighbourhood variables together. That's the route the lesson describes for moving beyond purely ecologic analysis, because it can separate the effect of a person's own characteristics from the effect of where they live.

**Sarah:** But there's a trap in the second one, and I think it's worth saying plainly. Calling it non-ecologic removes the cross-level problem, and confounding is still there. Provinces that commute actively may also be wealthier, younger or denser, and any of those could explain lower diabetes rates.

**Kiffer:** That's an important catch. Keeping the inference at the group level answers one objection, and the ordinary questions about confounding, measurement and chance still apply at the group level. A real Canadian example shows how researchers handle that. Calgary stopped fluoridating its water in 2011, while Edmonton continued. McLaren and colleagues examined Grade 2 children in both cities, and in 2018 and 2019 about sixty-five percent of the Calgary children had a history of caries in their baby teeth, compared with about fifty-five percent in Edmonton.

**Sarah:** And with only two cities, anything else that differs between Calgary and Edmonton is tangled up with fluoridation. So how did they deal with that?

**Kiffer:** They adjusted for individual factors such as brushing and flossing, other sources of fluoride, family socioeconomic status and ethnocultural background, and the difference between the cities remained. They also collected fingernail clippings from some of the children as a marker of fluoride intake. The exposure is a city policy, but the outcomes and the other factors were measured on individual children, so the comparison doesn't rest on city averages alone.

**Sarah:** This is where I want to push on something. I think the ecological fallacy gets invoked too quickly. In 1994, Schwartz wrote a paper called The Fallacy of the Ecological Fallacy. She argued that cross-level inference is a validity problem that individual-level studies have too, and that treating ecologic studies as inherently suspect discourages attention to group-level causes. Rose made a related point when he separated the causes of a case from the causes of incidence. Why a whole population has a high rate is a legitimate question, and group data are often the right data for it.

**Kiffer:** I agree with most of that, and I think Schwartz's paper is a fair correction. Where I'd push back is on how often the warning is needed in practice. When Dufault and Klar reviewed recent cross-sectional ecologic studies in major epidemiology journals, about half left out important information about whether the research was susceptible to the ecological fallacy, and more than half left out important information about why the design was chosen. My reading is that many published ecologic studies drift from group-level data into individual-level language, so I'd keep raising the question.

**Sarah:** I'd read those same numbers as a reporting problem. They show that authors often fail to say what level their question is at, which is a separate matter from whether group-level questions are worth asking.

**Kiffer:** That's fair, and I think we're closer than it sounds. My concern is the drift in the conclusions, and your concern is that the warning gets used to dismiss group-level work altogether.

**Sarah:** So the practical rule is to identify the level of the question first. If the conclusion is about groups, judge the study as a group-level study, with all the usual concerns about confounding. If the conclusion is about individuals, ask what individual-level evidence supports it.

**Kiffer:** Agreed, and the rule cuts both ways. Carrying an individual-level finding up to explain group rates is the atomistic fallacy, and herd immunity is the standard reminder that populations have properties no single person has. The modifiable areal unit problem from the lesson adds one more check for both kinds of study. Ask how the groups were drawn, because the same people grouped differently can give a different area-level result.

**Sarah:** Let's pull it together. First, the central limitation is the missing joint distribution. The same group totals can come from a world where the exposure is harmful or a world where it's protective, as our two made-up towns showed.

**Kiffer:** Second, the group-level rate ratio compares a fully exposed group with a fully unexposed group, which almost always means extrapolating the fitted line beyond the data, and misclassification of exposure tends to inflate that ratio away from the null.

**Sarah:** And third, a study that uses group data is ecologic only when it draws conclusions about individuals. Keeping the conclusion at the group level avoids the cross-level fallacies, and the usual questions about confounding still apply.

**Kiffer:** If you'd like more practice, rework the misclassification example from this episode with a sensitivity and specificity of eighty percent each, and check your answer with the lesson's formula.

**Sarah:** Next time, it's Lesson seven, Conceptualization, Measurement and Causal Specification, where we look at how the exposures, outcomes and confounders in a study are defined and measured in the first place.

**Kiffer:** Take care, everyone.

**Sarah:** See you in Lesson seven.
