# Lesson 7: Conceptualization, Measurement & Causal Specification

*Companion-podcast transcript • Sarah & Kiffer*  
*Office Hours episode to listen to after working through the lesson*

---

**Sarah:** Welcome back to Office Hours. I'm Sarah.

**Kiffer:** And I'm Kiffer. This episode goes with Lesson seven, Conceptualization, Measurement and Causal Specification.

**Sarah:** It's meant for after you've worked through the lesson. We'll add some perspective on how these ideas show up in real studies and some critique of the methods and conventions, work through the questions students tend to find thorny with this material, and do some extra worked examples, including a couple that are harder than the ones on the lesson page.

**Kiffer:** Along the way there are a few places where we ask you to work something out before we give the answer. When that happens you'll hear a few seconds of quiet, and you can pause the audio if you'd like more time to think it through.

**Sarah:** We have five questions. First, what random measurement error does to an association, and how to correct for it. Second, whether the same score means the same thing in different groups. Third, whether a Likert answer can be treated as a number, which is where Kiffer and I disagree a little. Fourth, why adding a variable to a model can make an estimate worse. And fifth, what a lag analysis can and can't tell you about reverse causation.

**Kiffer:** Question one. A measure can be reliable and still miss its construct, like a bathroom scale that reads two kilograms high for everyone. The opposite problem gets less attention. A measure can aim at the right construct and still bounce around from one occasion to the next. Sarah, what happens to an association when the exposure carries a lot of random error?

**Sarah:** It gets pulled toward the null. If I measure salt intake with a single food frequency questionnaire, some people with high intake will look moderate, and some people with low intake will look moderate too. The real spread gets blurred, so the slope relating salt to blood pressure flattens out.

**Kiffer:** Right. In the simplest case, one continuous exposure with purely random error that's the same for everyone, the observed slope equals the true slope times the reliability. Reliability here is the share of the variance in the measured exposure that reflects real differences between people. Here's a made-up example. A hypothetical cohort measures daily sodium with one questionnaire and finds that each extra gram a day goes with a systolic blood pressure 1.5 millimetres of mercury higher. A validation substudy, in which some participants also gave repeated urine samples, puts the reliability of the questionnaire at 0.5. What's the corrected slope? Take a few seconds.

*(Pause)*

**Sarah:** The reliability is 0.5, so I multiply. 1.5 times 0.5 is 0.75. So the corrected slope is 0.75 millimetres of mercury per gram.

**Kiffer:** Before we go on, which way should a correction for attenuation move the estimate, toward the null or away from it?

**Sarah:** Away from it. Attenuation made the observed slope too small, so the corrected slope has to be bigger than 1.5, and I made it smaller. I applied the attenuation a second time. The observed slope is the true slope times 0.5, so the true slope is the observed slope divided by 0.5. That's 1.5 divided by 0.5, which is 3.

**Kiffer:** That's it, three millimetres of mercury per gram. If the reliability had been 0.6, the corrected slope would be 1.5 divided by 0.6, which is 2.5. The lower the reliability, the bigger the correction. That's the idea behind regression calibration, and it's why validation substudies matter so much. Without some estimate of the reliability, you can say the true slope is probably steeper, but you can't say by how much.

**Sarah:** I want to add a caution, because people sometimes treat attenuation as a guarantee. Random error that's the same in the groups being compared pulls a single exposure's estimate toward the null. Once the error is differential, as when cases remember past exposures more completely than controls, the bias can go either way. And when the poorly measured variable is a confounder, the error leaves some of the confounding behind, which is the residual confounding from Section three.

**Kiffer:** That last point matters a great deal in nutrition research, where diet, smoking and income are all measured with error at the same time. A null finding from a single questionnaire might mean there's no effect, or it might mean the measure was too noisy to show one. Reporting the reliability, or at least citing a validation study, lets readers judge which.

**Sarah:** Question two. If two groups get the same score on a scale, does that score mean the same thing in both? I'll set up a made-up example. Imagine a hypothetical twenty-item depression scale, with each item scored from zero to three and a cutoff of sixteen. Four of the items are about the body: restless sleep, poor appetite, low energy, and feeling that everything is an effort. Now picture two groups of older adults with the same level of underlying depression, who answer the mood items in much the same way. One group is healthy. The other group is recovering from hip surgery.

**Kiffer:** And recovery from surgery affects sleep, appetite and energy whatever a person's mood happens to be, so those four items will pick up the surgery itself.

**Sarah:** Exactly. Suppose the healthy group averages twelve points, and the recovery itself adds about one point to each of the four bodily items in the surgery group. What's that group's average, and what would a researcher who compared the two groups conclude? Take a few seconds.

*(Pause)*

**Kiffer:** Twelve plus four times one is sixteen, so the surgery group averages right at the cutoff. A researcher comparing the groups would report far more depression after surgery, even though in this made-up example the underlying depression is the same. Those four points are construct-irrelevant variance. They come from physical illness, and the scale counts them as depression.

**Sarah:** So what would a careful analyst check before believing a gap like that in real data?

**Kiffer:** Start by splitting the score. If the gap sits almost entirely in the bodily items and the mood items look alike, that's a warning sign. The formal version is a differential item functioning analysis, which compares how people in different groups answer each item after matching them on their overall level of the trait. Iwata and colleagues did this with the CES-D in 2002 and found that about half of its items worked differently for African American and Hispanic young adults than for non-Hispanic white young adults.

**Sarah:** Self-rated health raises the same issue with a single question. Statistics Canada asks it in many of its surveys, including the Canadian Community Health Survey. In general, how is your health? Excellent, very good, good, fair or poor.

**Kiffer:** It's a strong predictor of death, which is part of why it's so widely used. But what counts as good health depends on what a person compares themselves with. Franks and colleagues, using a large American survey and five years of follow-up, reported in 2003 that self-rated health was more strongly related to later mortality in younger people, in people with more education, and in white respondents. So a gap in self-rated health between two groups mixes real differences in health with differences in how people hear the question.

**Sarah:** And this connects back to the frameworks at the start of the lesson. Somebody chose these instruments, and that choice reflects a theory about what matters.

**Kiffer:** Right, and deciding which groups to compare is part of that theory. If a study never breaks its results down by income or by Indigeneity, it can't see a gap there. If it does break them down with an instrument that hasn't been checked for invariance, it may see a gap that's partly an artifact. Both choices deserve a sentence of justification in the methods.

**Sarah:** Question three is where we disagree. A Likert item has ordered categories, like strongly disagree up to strongly agree. Can you code it one to five and treat it as a number? Let me start with a worked example, again made up. A hypothetical survey asks adults to describe themselves as inactive, somewhat active, active or very active, and records their systolic blood pressure. With equal numbers in each group, the average blood pressures are 132, 126, 124 and 123.

**Kiffer:** So the steps between neighbouring categories are six points, then two, then one. The biggest change is from inactive to somewhat active, which fits the evidence that the largest health gains from activity come at the low end.

**Sarah:** Now code the categories one to four and fit a straight line. The fitted slope is a drop of 2.9 points per category. That line says the first step lowers blood pressure by 2.9 points, when the actual difference there is six, and it says the last step lowers it by 2.9 points, when the actual difference is one.

**Kiffer:** So the linear coding cuts the most important difference roughly in half and makes the smallest one nearly three times too big. Entering the categories as separate indicator variables would give each step its own estimate and show the curve directly.

**Sarah:** Here's my position, though. That example uses a single item. Most of the scales we use add up many items, and a sum of ten or twenty Likert items behaves much more like an interval measure, because the uneven steps of the individual items smooth out. A slope per point is also easy to explain to an interest holder. An ordinal regression gives you odds ratios for being in a higher category, which most audiences find hard to follow. For summed scales, I think treating the score as a number is usually reasonable.

**Kiffer:** I agree about summed scales, mostly. Where I'd push back is on single items, and on summed scales whose answers pile up at one end, which happens often with symptom scales in general population samples, where most people score near zero. Liddell and Kruschke's simulations found that treating ordinal data as metric can produce false positives, hide real effects, and even reverse the direction of an effect, and the risk is greatest when responses are skewed or the categories are unequally spaced. My view is that the convenience of a single slope isn't worth that risk unless you've checked.

**Sarah:** But checking everything with ordinal models has costs too. They can be harder to fit in small samples, and if the straight-line result and the ordinal result agree, you've spent effort to confirm what you already had.

**Kiffer:** That's fair, and agreement is useful information in itself. I'd treat it as a sensitivity analysis. Fit the simple version, then fit the categories as indicators, and see whether the conclusion changes.

**Sarah:** Then I think we land in the same place. For a single item or a skewed scale, look at the category-by-category estimates before trusting one slope. For a well-behaved summed scale, a linear model is usually fine. Either way, report which you did and whether the answer changed.

**Kiffer:** Question four. Why can adjusting for one more variable make an estimate worse? Here's a harder made-up example that combines several ideas. A hypothetical cohort of hospital staff asks whether working night shifts causes type 2 diabetes. The authors adjust for three things: age, household income in three broad bands, and weight gain measured two years after people started their shift pattern. And the analysis includes only staff who attended a voluntary workplace health screening. Sarah, let's take them one at a time.

**Sarah:** Age is the easy one. Younger staff are more likely to be on nights, and age affects diabetes risk, so age is a common cause of the exposure and the outcome. It's a confounder, and adjusting for it is right.

**Kiffer:** Now the weight gain measured two years after the shift pattern started. In this diagram, is that a confounder, a mediator or a collider? Take a few seconds.

*(Pause)*

**Sarah:** It's measured after the exposure began, and night work can plausibly disrupt sleep and mealtimes and lead to weight gain, which raises diabetes risk. So it sits on the pathway from shift work to diabetes. It's a mediator. If the question is about the total effect of shift work, adjusting for it blocks part of the effect we're trying to estimate, which is overadjustment.

**Kiffer:** Right, and timing is the clue. Body weight before anyone started nights could be a confounder. Weight gain afterward is downstream of the exposure, and the backdoor criterion excludes any descendant of the exposure from the adjustment set. What would you say about income?

**Sarah:** Income is a confounder. Lower-paid staff may take night shifts for the extra pay, and lower income is linked with diabetes. So it belongs in the model. But three broad bands lump together people in quite different circumstances.

**Kiffer:** So adjusting for it leaves residual confounding. The fix is to keep income and measure it better, as a continuous figure or with education and job grade alongside it, and to read the adjusted estimate as still partly confounded. Then there's the screening restriction, which is the trap in this example.

**Sarah:** I'd have assumed that restricting to the people who came to the screening just makes the sample smaller.

**Kiffer:** Think about who comes. Suppose the screening clinic runs during day hours, so day staff find it easier to attend. And suppose people with a family history of diabetes are more likely to attend because they're worried. Attendance is caused by shift pattern and by family history, so it's a collider between them. Among the night-shift staff who did make it in, what's likely to be true about family history?

**Sarah:** They needed a stronger reason to come, so more of them will have a family history. Within the screened sample, night work and family history become linked, even if they aren't linked in the whole workforce. And family history causes diabetes, so night work will look more harmful than it really is.

**Kiffer:** That's the collider path opening. In this made-up setup the bias pushes the estimate up, and in a real study the direction depends on who is drawn to the screening and why. The obesity paradox has the same structure. In a 2014 paper, Hailey Banack and Jay Kaufman at McGill University analysed American national survey data. In the whole sample, obesity went with a higher risk of death, with an adjusted risk ratio of 1.24. Among people with cardiovascular disease the risk ratio was 0.79, so obesity looked protective. They showed that conditioning on cardiovascular disease is one plausible explanation for the reversal.

**Sarah:** So a corrected analysis adjusts for age and a finer measure of income, leaves the weight gain out if the target is the total effect, and uses the whole workforce, or at least reports who attended the screening and why.

**Kiffer:** Exactly. The general habit is to draw the diagram first and give each variable a role before deciding what goes into the model. A list of available covariates is a different thing from an adjustment set.

**Sarah:** Question five. When a lag analysis makes an association weaker, what have we learned? Here's a real example. Strain and colleagues used self-reported physical activity in about 450,000 UK Biobank participants. With follow-up cut off after one year, the hazard ratio linking more activity with death from any cause was 0.73. Over seven years it was 0.86. Excluding people with disease at the start and deaths in the first two years gave 0.88. Why would the association get weaker as follow-up gets longer? Take a few seconds.

*(Pause)*

**Kiffer:** In the first year, many of the people who die were already sick when they filled in the questionnaire, and being sick is a reason to be inactive. So early deaths make inactivity look more deadly than it is. As follow-up lengthens, those early deaths become a smaller share of the total. In percentage terms, the hazard went from twenty-seven percent lower, to fourteen percent lower, to twelve percent lower.

**Sarah:** So the weakening fits reverse causation. And the fact that it settles around 0.88 suggests that some of the association survives.

**Kiffer:** It suggests some of it survives, though a lag analysis can't separate a real effect from residual confounding. People who stay active also tend to smoke less and have more money, and the lag does nothing about that. A lag removes one explanation and leaves the others on the checklist.

**Sarah:** Which brings in simultaneity. Luppino and colleagues pooled fifteen longitudinal studies and found that obesity at baseline raised the odds of later depression, with an odds ratio of 1.55, and depression at baseline raised the odds of later obesity, with an odds ratio of 1.58. Students often mix up reverse causation and simultaneity, so how would you separate them?

**Kiffer:** Reverse causation means we've drawn a single arrow pointing the wrong way. Simultaneity means arrows run both ways. In the Luppino studies each direction was seen over time, which is why longitudinal data could show it at all. A cross-sectional study that measures both at one moment can't separate the two directions, and calling one of them the exposure builds the wrong answer into the model. It's also worth noticing that those pooled odds ratios were unadjusted, so confounding is still on the list for both directions.

**Sarah:** So the practical question to ask of any study is whether its timing can tell the directions apart, and then what's left on the checklist after that.

**Kiffer:** Let's pull it together with three things worth taking away from this episode.

**Sarah:** First, random error that's the same across groups weakens associations, and a reliability estimate lets you correct for it by dividing the observed slope by the reliability. Without one, a null finding from a noisy measure is hard to interpret.

**Kiffer:** Second, check what a score means in each group before comparing groups, and state the theory behind your choice of instruments and of which groups to compare.

**Sarah:** And third, give every variable a role in a diagram before adjusting. Confounders go in, mediators stay out when the total effect is the target, conditioning on a collider opens a path, and a coarsely measured confounder leaves residual confounding behind.

**Kiffer:** For practice, rework the night-shift example with one change. Suppose the screening clinic ran at night, so night staff found it easier to attend, and work out which way the bias would go.

**Sarah:** Next time, it's Lesson eight, Sampling, Selection Processes and External Validity, where we look at who ends up in a study and what that does to its conclusions.

**Kiffer:** Thanks for working through these problems with us today. Take care, everyone.

**Sarah:** See you in Lesson eight.
