# Lesson 3: Introduction to Observational Studies

*Companion-podcast transcript • Sarah & Kiffer*  
*Office Hours episode to listen to after working through the lesson*

---

**Sarah:** Welcome back to Office Hours. I'm Sarah.

**Kiffer:** And I'm Kiffer. This episode goes with Lesson three, Introduction to Observational Studies.

**Sarah:** It's meant for after you've worked through the lesson. We'll bring in some perspective on how these designs get used in practice, some critique of where they fall short, a few of the questions students tend to find thorny, and some extra worked examples, including a couple that are harder than the ones on the lesson page.

**Kiffer:** There are also a few places where we'll ask you to work something out before we give the answer. When that happens, you'll hear a few seconds of quiet, and you can pause the audio if you'd like more time to think it through.

**Sarah:** Here's what we'll cover. First, what the hierarchy of evidence can and can't tell you, which is where Kiffer and I disagree a little. Second, the prevalence ratio and the prevalence odds ratio, and why they drift apart. Third, which exposures a cross-sectional study handles well. Fourth, a harder problem about how a factor that only lengthens an illness can look like a cause of it. And fifth, how to get incidence out of two snapshots.

**Kiffer:** Question one. If a cohort study sits above a cross-sectional study in the hierarchy, should we always trust the cohort more? I'd like to start with a real story that shows why the answer is complicated.

**Sarah:** Go ahead. I suspect it's the story about hormone therapy after menopause.

**Kiffer:** It is. For years, large observational studies, including the Nurses' Health Study, a big American cohort, suggested that women who used hormone therapy after menopause had less coronary heart disease. One of the later analyses put the hazard ratio for current users of estrogen plus progestin, compared with never users, at about 0.68. Then the Women's Health Initiative randomized trial found more coronary heart disease in the hormone arm than in the placebo arm, about sixty-eight percent more in the first two years.

**Sarah:** So a cohort study, which is supposed to be near the top of the hierarchy, pointed in the opposite direction from the trial. That sounds like a point for people who say observational studies can't be trusted.

**Kiffer:** That was a common reaction. In 2008, Miguel Hernán and colleagues reanalysed the Nurses' Health Study data as if it were a trial. They emulated a target trial. They took women who were eligible at a given time, compared those who started hormone therapy at that time with those who didn't, and began follow-up at that same moment. Their intention-to-treat estimates were similar to the trial's.

**Sarah:** So what had gone wrong in the original analyses?

**Kiffer:** A large part of it was who counted as exposed. The original analyses compared current users with never users. Current users are women who started therapy at some point and stayed on it, so the early period after starting, when the trial saw the extra risk, was mostly missing. The authors also found that differences in time since menopause and in length of follow-up explained much of the gap between the two studies.

**Sarah:** That sounds a lot like the problem with prevalent cases in a cross-sectional study. If you count current users, you count the people who stuck with the exposure, the same way a snapshot counts the people whose disease has lasted.

**Kiffer:** That's the link I wanted to draw. Studying prevalent exposure carries the same trap as studying prevalent disease. The target trial protocol from the lesson prevents it by lining up eligibility, the start of treatment and the start of follow-up at one time zero.

**Sarah:** Which brings me to where we disagree. I think the hierarchy of evidence is still a good default for students. When you know nothing else about two studies, the one with the stronger design should get more weight.

**Kiffer:** I'd put more emphasis on purpose. Vandenbroucke argued in 2008 that the hierarchy ranks designs for one job, evaluating the effect of something like a treatment. For discovery, case reports and case series often come first. The 1981 report of five young men in Los Angeles with Pneumocystis pneumonia was the first published sign of what became the AIDS epidemic. And Concato and colleagues found in 2000 that well-designed cohort and case-control studies gave average results remarkably similar to trials of the same clinical questions. So I think how a study handles time zero, confounding and measurement tells you more than its label.

**Sarah:** I agree with all of that for someone who can already appraise those things. A reader who's new to this and can't yet judge time zero needs a starting point, and a ranking by design is a defensible one. The hormone story also shows that the fix came from better design, which supports the idea that design matters.

**Kiffer:** That's fair, and I'd keep the hierarchy as the first question a reader asks. My worry is when it becomes the last question, so that a study is accepted or dismissed because of its label alone.

**Sarah:** Then here's a rule we can both live with. Use a design's place in the hierarchy to decide which questions to ask first, and then judge the specific study on how it chose its people, lined up its timing and handled confounding.

**Kiffer:** Agreed. That rule also explains why it's worth learning the cross-sectional design in detail, because its weaknesses are specific and a careful reader can check for each of them.

**Sarah:** Question two is the one I want to try myself. Here's a made-up survey. A university surveys four hundred students at one point in time. Two hundred say they usually sleep less than six hours a night, and sixty of them score high on an anxiety scale. Of the two hundred who sleep longer, thirty score high. I'll work out both measures of association.

**Kiffer:** Before Sarah starts, try it yourself. What are the prevalence ratio and the prevalence odds ratio here, and which one would you report to a general audience? Take a few seconds.

*(Pause)*

**Sarah:** The prevalence among the short sleepers is sixty out of two hundred, which is thirty percent. Among the longer sleepers it's thirty out of two hundred, which is fifteen percent. So the prevalence ratio is thirty divided by fifteen, which is two.

**Kiffer:** Good, that's the prevalence ratio. Now walk us through the prevalence odds ratio in the same way.

**Sarah:** The odds among short sleepers are sixty divided by one hundred and forty, about 0.43. Among longer sleepers, it's thirty divided by one hundred and seventy, about 0.18. Divide the first by the second and you get about 2.43. So I'd tell people that high anxiety is about two and a half times as common among short sleepers.

**Kiffer:** Compare that sentence with the two prevalences you found a moment ago. Does two and a half times as common match thirty percent compared with fifteen percent?

**Sarah:** No, it doesn't. Thirty percent is exactly twice fifteen percent. The 2.43 is a ratio of odds, and I described it as if it were a ratio of prevalences. When the outcome is common, the odds ratio sits further from one than the prevalence ratio, so reading it as a prevalence ratio overstates the association.

**Kiffer:** Right. With an outcome as common as thirty percent, the gap is noticeable. If the outcome were rare, say two percent compared with one percent, the two measures would be almost identical, about 2.02 for the odds ratio and exactly two for the prevalence ratio. For a general audience, the prevalence ratio of two is the clearer number.

**Sarah:** And before anyone reads either number as a cause, there's an obvious problem. Anxiety can keep people awake at night. The snapshot can't tell us whether short sleep came first.

**Kiffer:** That's reverse causation, and it leads straight into our next question about which exposures a snapshot can handle safely.

**Sarah:** Question three. Cross-sectional studies are said to be best suited to time-invariant exposures, such as sex. I think students sometimes stretch that and assume that any exposure from the past is safe to study this way.

**Kiffer:** Let's test that with a hypothetical survey of adults aged fifty to seventy, all measured once. It finds that knee pain is associated with three exposures: how many hours a week people say they're active now, their adult height, and whether they say they grew up on a farm. For each one, how worried should we be that the outcome shaped the exposure, or shaped how the exposure was reported? Take a few seconds.

*(Pause)*

**Sarah:** Current activity is the worst. Knee pain makes people less active, so the arrow could easily run from the outcome to the exposure. Height looks safest, because adult height was set long before anyone developed knee pain in middle age.

**Kiffer:** And what about growing up on a farm, which happened decades before anyone in the survey had knee pain?

**Sarah:** That one's tricky. The exposure itself happened long ago, so knee pain can't have caused it. It's reported from memory, though, and someone with chronic pain might think harder about the physical work they did as a child. So the report could still differ by outcome.

**Kiffer:** That's the distinction I wanted. A fixed exposure protects you from reverse causation in the exposure itself. It gives no protection from recall bias in how the exposure is measured. Height measured with a tape is better than height recalled, and a childhood address taken from records is better than one taken from memory.

**Sarah:** The postpartum depression study in the lesson has the same issue. Stress during pregnancy was one of the strongest factors, and it was reported after the birth.

**Kiffer:** Yes. Lanes and colleagues analysed 6,421 women from the Maternity Experiences Survey, who were interviewed between five and fourteen months after giving birth. Pregnancy is over by then, so the stress can't have been caused by later depression. A woman with depressive symptoms might still remember her pregnancy as more stressful, so the timing of the exposure is settled while the accuracy of the report is open to question.

**Sarah:** So a time-invariant exposure is necessary for the safest case, and how it's measured still matters.

**Kiffer:** Yes. There's also a second problem that a fixed exposure doesn't solve at all, and that's our harder example.

**Sarah:** Question four is the harder one. It combines a time-invariant exposure with the link between prevalence, incidence and duration.

**Kiffer:** Here's a made-up example. Imagine a gene variant that people are born with. A cross-sectional study of a chronic lung disease finds the variant is more common among people who have the disease. The disease can't cause the variant, so reverse causation is ruled out. Now suppose, hypothetically, that the true incidence is five new cases per one thousand person-years in carriers and non-carriers alike. The only difference is that carriers, once they're sick, live with the disease for an average of eight years, while non-carriers live with it for four.

**Sarah:** So the variant changes how long people have the disease and leaves their chance of getting it unchanged.

**Kiffer:** That's the setup. Using the approximation that prevalence is roughly incidence times duration, work out the prevalence in each group and the prevalence ratio. Then decide whether the study has found a cause. Take a few seconds.

*(Pause)*

**Sarah:** Five per one thousand person-years is 0.005 per year. For non-carriers, 0.005 times four years is 0.02, so about two percent. For carriers, 0.005 times eight is 0.04, about four percent. The prevalence ratio is about two.

**Kiffer:** And if you use the more exact steady-state formula, prevalence equals incidence times duration divided by one plus incidence times duration, you get 0.0196 and 0.0385, which gives a prevalence ratio of 1.96. Either way it's about two.

**Sarah:** So the study has found a real difference, and what it has found is a factor that lengthens survival with the disease. The variant has no effect on who develops it.

**Kiffer:** Right, and there's a tidy way to see the size of the problem. Under that steady-state formula, the prevalence odds equal incidence times duration exactly. So the prevalence odds ratio equals the incidence rate ratio times the ratio of the durations. Here the incidence rate ratio is one and the duration ratio is two, so the prevalence odds ratio is exactly two, and all of it comes from duration.

**Sarah:** Which means a variant that helps people live longer with the disease could be reported as a risk gene for it.

**Kiffer:** It could. The bias can also run the other way. If a variant shortened survival, carriers would leave the pool of cases quickly, and a snapshot would make the variant look protective. The remedy is to study new cases, which the case-control and cohort designs in the next two lessons can be built to do.

**Sarah:** So for this kind of question, a cross-sectional study can't separate a factor that causes the disease from one that keeps people alive with it, even when the exposure is fixed at birth.

**Kiffer:** Unless you have outside information on duration, that's right. A survey can still be very useful for describing burden. The Ontario survey on the prevalence and control of hypertension examined 2,551 adults and found that about twenty-one percent had hypertension, which is the kind of number a health system needs for planning.

**Sarah:** Question five. There are three ways to get incidence out of cross-sectional data, and I'd like to work through the mathematical one with numbers.

**Kiffer:** Suppose, hypothetically, that a survey finds ten percent of forty-year-olds and nineteen percent of fifty-year-olds have a lifelong condition. Equation 3.1 in the lesson works from the share of people who are still free of the condition. At forty that's ninety percent, and at fifty it's eighty-one percent.

**Sarah:** So the healthy share drops from 0.90 to 0.81. The fraction of the healthy group that became sick is the change in prevalence, 0.09, divided by 0.90, which is 0.10. Over ten years, ten percent of the healthy forty-year-olds developed the condition.

**Kiffer:** And how would you turn ten percent over ten years into an average yearly figure?

**Sarah:** My first instinct is one percent a year. The formula says one minus the quantity one minus 0.10, raised to the power of one over ten. Ninety percent to the power of one tenth is about 0.9895, so the yearly figure is about 0.0105, a little over one percent.

**Kiffer:** The two answers are close here because the rate is low. The formula allows for the healthy group shrinking a little each year, which matters more when incidence is high. The bigger issue is the set of assumptions behind it.

**Sarah:** I can think of three. Nobody recovers, people with the condition don't die or move away faster than people without it, and the forty-year-olds and the fifty-year-olds are comparable apart from age.

**Kiffer:** The third one is easy to miss. If the two age groups come from one survey, they're different people born ten years apart. If the condition has become more common in younger generations, the comparison mixes aging with change across birth cohorts. And if people with the condition die earlier, the prevalence at fifty is too low, and the incidence comes out too low as well.

**Sarah:** Which is where repeated cross-sections come in. Statistics Canada's Canadian Community Health Survey has collected data every year since 2007 from people aged twelve and over, with a fresh cross-sectional sample each time.

**Kiffer:** That design suits questions about how the population is changing, because each cycle represents the population at that time. What it can't show is who moved from healthy to sick. For that you need a cohort that follows the same people, with the costs of dropout and a group that becomes more selected over time. HIV researchers developed the third approach, two tests on one blood sample. People who are positive on a standard test and negative on a less sensitive one are classed as recently infected, which Janssen and colleagues described in 1998.

**Sarah:** So each method gets you incidence at a price. One needs a model with assumptions, one needs a second survey, and one needs a second test whose window for recent infection is known.

**Kiffer:** Let's pull it together with three things to take away from this episode.

**Sarah:** First, a design's place in the hierarchy tells you which questions to ask first. Whether a particular study deserves trust depends on how it handled selection, time zero and confounding.

**Kiffer:** Second, a cross-sectional study counts existing cases, and existing cases reflect both how often a disease starts and how long it lasts. A factor that only lengthens illness, even one people are born with, can look like a cause.

**Sarah:** And third, check which ratio you're reading. With a common outcome, the prevalence odds ratio sits further from one than the prevalence ratio, so describe each one in its own terms.

**Kiffer:** If you'd like more practice, rework the gene variant problem with a variant that halves the duration of the disease, and work out what the prevalence odds ratio would show.

**Sarah:** Next time, it's Lesson four, Case-Control Studies, where we start from people who have the disease and look back at their exposures.

**Kiffer:** Take care, everyone.

**Sarah:** See you in Lesson four.
