# Lesson 4: Case-Control Studies

*Companion-podcast transcript • Sarah & Kiffer*  
*Office Hours episode to listen to after working through the lesson*

---

**Sarah:** Welcome back to Office Hours. I'm Sarah.

**Kiffer:** And I'm Kiffer. This episode goes with Lesson four, Case-Control Studies.

**Sarah:** It's meant for after you've worked through the lesson. We'll bring in some perspective on how case-control studies have been used, add some critique of where they go wrong, work through the questions students tend to find thorny, and do some extra worked examples, including a couple that are harder than the ones on the lesson page.

**Kiffer:** There are a few places where we'll ask you to work something out before we give the answer. When that happens, you'll hear a few seconds of quiet. Pause the audio if you'd like more time to think it through.

**Sarah:** Here's the plan. Question one asks what an odds ratio from a case-control study actually estimates. Question two asks who belongs in the control group, with two famous studies from history. Question three works through how recall bias moves an odds ratio. Question four takes up the case-crossover design and its question of why now. And then we'll disagree about whether case-control studies deserve their reputation as weaker evidence than cohort studies.

**Kiffer:** Question one. When a case-control study reports an odds ratio, what is that number telling you? I'd like to start with a made-up outbreak, because outbreaks are where the answer surprises people. Imagine a community potluck with three hundred guests. Two hundred ate the chicken salad, and one hundred and twenty of them got sick. One hundred people skipped it, and twenty of them got sick.

**Sarah:** So a hundred and forty people got sick in total, and a hundred and sixty stayed well.

**Kiffer:** Right. Now suppose the health unit runs a case-control study. It takes all one hundred and forty sick guests as cases, and it samples one hundred and forty of the well guests as controls. Because the controls are sampled without regard to what they ate, about seventy of them ate the salad and about seventy didn't. Among the cases, one hundred and twenty ate it and twenty didn't. Here's the question for everyone. What's the odds ratio, and what would you tell a reporter it means? Take a few seconds.

*(Pause)*

**Sarah:** Let me try this one. The cross-product is one hundred and twenty times seventy, which is eight thousand four hundred, divided by seventy times twenty, which is one thousand four hundred. That gives an odds ratio of six. So I'd tell the reporter that people who ate the salad had six times the risk of getting sick.

**Kiffer:** In this outbreak we happen to know everyone's outcome, so we can check that. What was the risk among the guests who skipped the salad?

**Sarah:** Twenty out of a hundred, so 0.2. And six times 0.2 is 1.2. That would mean a hundred and twenty percent of salad eaters got sick, which is impossible. So the odds ratio can't be read as six times the risk here.

**Kiffer:** Good, that's exactly the sanity check to run. So what's the actual risk ratio?

**Sarah:** The risk among salad eaters is one hundred and twenty out of two hundred, which is 0.6. Divided by 0.2, that's a risk ratio of three. So the odds ratio of six is twice the risk ratio.

**Kiffer:** And the reason is that the illness was common. About forty-seven percent of the guests got sick. In a risk-based design like this one, where the controls are the people still well at the end, the odds ratio approximates the risk ratio only when the outcome is rare in the source population. When it's common, the odds ratio sits farther from one than the risk ratio does.

**Sarah:** And when the disease is rare, the two numbers come together. If the risks were six in a thousand compared with two in a thousand, the risk ratio would be three and the odds ratio would be about 3.01.

**Kiffer:** That's the rare-disease assumption at work. I'd add one thing that students often miss. In a rate-based design, where controls are sampled from the people still at risk at the moment each case occurs, the odds ratio estimates the incidence rate ratio, and no rarity assumption is needed. So the honest reply to the reporter is a question back. How were the controls sampled, and how common is the outcome?

**Sarah:** Let me add that the odds ratio of six is still a correct number. It's a valid measure of association in its own right. The error is only in calling it a risk ratio.

**Kiffer:** Question two. Who belongs in the control group? The principle is that controls should represent the exposure experience of the population that gave rise to the cases. A good test is to ask, if this control person had developed the disease, would they have ended up as a case in my study?

**Sarah:** That test sounds simple, but I think people still picture controls as healthy volunteers.

**Kiffer:** They do, and one of the most famous early case-control studies is a good place to see why that picture is wrong. In their 1950 study of smoking and lung cancer, Richard Doll and Austin Bradford Hill took patients with lung cancer from hospitals and, for each one, chose a control patient of the same sex and five-year age group, in the same hospital, at about the same time. The controls had other diseases. They weren't healthy people at all.

**Sarah:** And the numbers from that study are worth working through, because they're so lopsided. Among the six hundred and forty-nine men with lung cancer, six hundred and forty-seven smoked and two did not. Among the six hundred and forty-nine male controls, six hundred and twenty-two smoked and twenty-seven did not. If we set aside the matching for a moment, what's the odds ratio? Take a few seconds.

*(Pause)*

**Kiffer:** The cross-product is six hundred and forty-seven times twenty-seven, which is seventeen thousand four hundred and sixty-nine, divided by two times six hundred and twenty-two, which is one thousand two hundred and forty-four. That gives about fourteen.

**Sarah:** Which matches the odds ratio of about fourteen that the lesson mentions. And notice where the information comes from. Almost every man in both groups smoked. The whole estimate rests on two non-smoking cases compared with twenty-seven non-smoking controls.

**Kiffer:** That's the sparse-cell point from the lesson, in a real study. If you work out the confidence interval on the log scale and ignore the matching, it runs from about three to about sixty. That's a wide interval, and nearly all of its width comes from that cell of two.

**Sarah:** So the association was clearly real, but its size was poorly pinned down. There's also the question of the controls themselves. Smoking causes plenty of diseases other than lung cancer, so hospital patients probably smoked more than the general population of men they were meant to represent.

**Kiffer:** Right, and that would push the odds ratio toward one. If anything, the hospital controls probably made the association look weaker than it was. That's a case where a flawed control group made the finding harder to see. The opposite can happen too. In 1981, Brian MacMahon and colleagues at Harvard reported in the New England Journal of Medicine that coffee drinking was associated with pancreatic cancer, with relative risks of about 1.8 for up to two cups a day and 2.7 for three or more.

**Sarah:** And that one didn't hold up.

**Kiffer:** It didn't. The controls were hospital patients, and critics pointed out that they were heavily weighted toward people with digestive disorders. If those conditions led people to cut back on coffee, the controls would understate coffee drinking in the study base, and coffee would look more common among the cases. MacMahon acknowledged that leaving out those controls reduced the relative risks. When his own group studied the question again in 1986, they did not confirm the association.

**Sarah:** So the same kind of control group pushed one study toward the null and the other away from it. What decides the direction is whether the controls' illnesses are linked to the exposure.

**Kiffer:** That's the practical lesson. For every control source, ask whether the reason these people are available as controls is related to the exposure you're studying.

**Sarah:** Question three. How does recall bias move an odds ratio? This is the harder worked example, because it combines the odds ratio with misclassification. I'll set it up with made-up numbers. Suppose that in the study base, twenty percent of people without the disease were truly exposed, and the true odds ratio is two.

**Kiffer:** So the odds of exposure among controls are twenty divided by eighty, which is 0.25. Doubling that gives odds of 0.5 among the cases, and odds of 0.5 means one exposed case for every two unexposed. So one third of the cases were truly exposed.

**Sarah:** Now we interview everyone about the exposure. Nobody reports an exposure they didn't have, but people forget some of the exposures they did have. In scenario A, cases and controls both remember eighty percent of their true exposures. In scenario B, the cases search their memories harder and remember ninety-five percent, while the controls still remember eighty percent. Before we calculate, which direction does each scenario push the odds ratio away from two? Take a few seconds.

*(Pause)*

**Kiffer:** Let's work scenario A. The controls report exposure at eighty percent of twenty percent, which is sixteen percent. The cases report eighty percent of one third, which is about 26.7 percent. The odds among cases are 0.267 divided by 0.733, about 0.364. The odds among controls are 0.16 divided by 0.84, about 0.190. The ratio is about 1.9.

**Sarah:** So equal forgetting in both groups pulls the odds ratio a little toward one, from two down to about 1.9.

**Kiffer:** Now scenario B. The controls still report sixteen percent. The cases report ninety-five percent of one third, which is about 31.7 percent. Their odds are 0.317 divided by 0.683, about 0.463. Divided by 0.190, that's about 2.4.

**Sarah:** So when cases remember better, the odds ratio is pushed above its true value, to about 2.4. A true doubling of the odds now looks like nearly two and a half.

**Kiffer:** And if the true odds ratio had been one, the same difference in recall would have produced an odds ratio above one from nothing. That's why differential misclassification worries people so much. It can create an association, and it can move an estimate in either direction depending on who reports more completely.

**Sarah:** The lesson's fictional night-shift study has exactly this problem, with an interviewer who knew who the cases were. What helps in practice?

**Kiffer:** Blinding the interviewers to case status where you can, asking about specific, well-defined exposures in the same way for everyone, and checking a sample of answers with records such as employer files. Some studies pick controls who've also had a recent illness, so that both groups have a similar reason to think hard about their past.

**Sarah:** Question four. The case-crossover design asks why now, where the usual case-control study asks why me. Can you give a real example of that?

**Kiffer:** A well-known Canadian one. In 1997, Donald Redelmeier and Robert Tibshirani studied six hundred and ninety-nine drivers in Toronto who had a cellphone and had been in a collision. For each driver, they compared phone use in the ten minutes before the crash with phone use in the same ten minutes on the day before. Each driver was their own control. Phone use was linked to about four times the risk of a collision, and hands-free phones showed no safety advantage over hand-held ones.

**Sarah:** Let me set up a made-up version so we can see how the arithmetic works. Imagine two hundred crashes, each compared with one control window. In fifteen, the driver was on the phone in both windows. In one hundred and thirty-five, they weren't on the phone in either. In forty, they were on the phone only in the window just before the crash, and in ten, only in the control window. What's the odds ratio? Take a few seconds.

*(Pause)*

**Kiffer:** It's forty divided by ten, which is four. The hundred and fifty drivers who were exposed the same way in both windows drop out. They can't tell you anything about whether the phone triggered the crash. Only the discordant pairs carry information, which is the logic of McNemar's test and the matched odds ratio.

**Sarah:** That surprised me the first time I saw it. Three quarters of the data simply don't contribute to the estimate.

**Kiffer:** They don't, and it's the price of the design's main strength. Because each driver is compared with themselves, everything stable about that person is controlled automatically. Their driving skill, their temperament, their eyesight, their usual route, all of it.

**Sarah:** But anything that changes from one day to the next isn't controlled. If people make more calls when they're running late, and running late also makes them drive badly, the phone gets the blame for the hurry.

**Kiffer:** Right, so the design trades between-person confounding for within-person confounding by time-varying factors. It also needs a transient exposure and an acute outcome, and if any control windows come after the event, the event mustn't change the exposure. A phone call before a crash fits, with a control window on the day before. Something like long-term diet would fit none of this.

**Sarah:** Now for the part where we disagree. Many evidence pyramids put case-control studies below cohort studies. I think that ranking is mostly deserved. Most case-control studies collect exposure information after the disease has happened, so recall bias is always a risk. The control group has to be built by the investigator, which is where the coffee study went wrong. And in a cohort, the exposure is measured before anyone knows who'll get sick.

**Kiffer:** I'd push back on that, at least in part. I think the label tells you less than the design details do. A nested case-control study inside a cohort uses exposure measured at baseline, often from stored blood samples, and its controls come from the same cohort. It's about as valid as the cohort it sits in, and far cheaper. And for a rare disease, a cohort may simply not be possible. Nobody could have followed enough pregnancies to find the link between DES taken in pregnancy and a rare vaginal cancer in the daughters. A case-control study of eight young women found it in 1971.

**Sarah:** That's a fair example, and Doll and Hill is another. But those successes involved very large associations. For the modest odds ratios we see most often now, say 1.3 or 1.5, the biases we just worked through are big enough to produce the whole effect. In scenario B, recall alone took two to about 2.4.

**Kiffer:** I agree that small odds ratios from retrospective interviews need real caution. Where I'd differ is on blaming the design as a whole. A cohort study with heavy loss to follow-up, or exposure measured once by a vague questionnaire, can be just as misleading. I'd rather people judged each study on its study base, how its controls were chosen and when its exposure was measured.

**Sarah:** I'm still more cautious than Kiffer about a modest odds ratio from an interview study. But we agree on the practical rule. Before you rank a study by its design label, check the study base, how the controls were sampled and whether exposure was recorded before the outcome was known.

**Kiffer:** Let's pull it together with three things to take away.

**Sarah:** First, an odds ratio from a case-control study is a valid measure, and what it estimates depends on how the controls were sampled. It approximates the risk ratio only for a rare outcome in a risk-based design, and it estimates the rate ratio with density sampling.

**Kiffer:** Second, controls should represent the exposure experience of the study base, and the reason they're available as controls must be unrelated to the exposure. Hospital controls can bias an odds ratio toward one or away from it.

**Sarah:** And third, when cases and controls report their exposures with different completeness, the odds ratio can move in either direction, and it can even appear from nothing. Blinded and standardized exposure assessment is the main defence.

**Kiffer:** If you'd like more practice, rework the recall bias example from this episode with a true odds ratio of one, and see how large an odds ratio the difference in recall produces on its own.

**Sarah:** Next time, it's Lesson five, Cohort Studies, where we follow exposed and unexposed people forward in time and measure incidence directly.

**Kiffer:** Take care, everyone.

**Sarah:** See you in Lesson five.
