HSCI 410 · Lesson 5

Modelling Dependent Data

Exploratory Data Analysis For Epidemiology

Learning objectives for this lesson:

  1. Identify clustered and repeated-measures data from the study design, name the variable that defines the clusters, and explain why the output of an ordinary regression gives no warning of clustering.
  2. Explain why ignoring clustering makes standard errors, confidence intervals and p-values too small, and distinguish cluster-level from person-level predictors to say which results are affected most.
  3. Interpret the intracluster correlation coefficient (ICC), the design effect and the effective sample size from R output, and explain why a clustered study gains more from extra clusters than from extra people per cluster.
  4. Interpret a linear mixed model with a random intercept fitted with lmer(): its fixed effects, its cluster and residual variances, the adjusted ICC and the estimated shift of a single cluster, compared with an ordinary regression.
  5. Describe the checks of a linear mixed model (residual plots, a Q-Q plot of the cluster effects, the number of clusters and a test for a singular fit), and explain what a random slope adds and what a singular-fit or convergence warning means.
  6. Interpret odds ratios from a logistic mixed model (GLMM, glmer()) and from GEE (geeglm()), distinguish cluster-specific from population-averaged odds ratios, and choose between the two methods from the research question and the number of clusters.
  7. Interpret a repeated-measures analysis in long format, including the group-by-time interaction in a mixed model or a GEE model, and explain how missed visits affect a complete-case analysis, a mixed model and GEE.

This course was developed by Dr. Kiffer G. Card based on Dohoo, I. R., Martin, S. W., & Stryhn, H. (2012). Methods in Epidemiologic Research. VER Inc.

Reference

Glossary: Key Terms, People & Concepts

📚 Reference page, available throughout the lesson

This glossary collects the key concepts, methods and people in this lesson. It can be used as a reference while working through the material or as a review before assessments. Typing in the search box filters the entries.

Key Concepts & Ideas
Independence Independence is the assumption that one observation tells us nothing about another. It fails when observations share a cluster, such as patients in the same clinic or visits by the same person.
Dependent Data Dependent data are data in which some observations are more alike than others because they share a cluster. Clustered data and repeated measures are the two common forms.
Cluster A cluster is the unit that groups observations together, such as a clinic, a school, a neighbourhood or a person who is measured several times. The cluster variable identifies which cluster each observation belongs to.
Clustered Data Clustered data arise when people are grouped inside larger units, such as patients in clinics or students in schools. Such data are also called multilevel or hierarchical data.
Repeated Measures Repeated measures are measurements of the same outcome on the same person at several times. Each person forms a cluster of visits, and studies of this kind are also called longitudinal studies.
Cluster-Level Predictor A cluster-level predictor takes the same value for everyone in a cluster, such as whether a clinic is urban. Its standard error is the most affected when clustering is ignored.
Person-Level Predictor A person-level predictor can differ between people in the same cluster, such as age or smoking. Its standard error is usually less affected by clustering.
Long and Wide Format In long format a repeated-measures dataset has one row per visit, so each person appears on several rows. In wide format it has one row per person with a column for each visit. Mixed models and GEE in R need long format.
Missing at Random (MAR) Data are missing at random when the chance that a value is missing depends only on information the model uses, such as a person's study arm or earlier measurements. Mixed models that use every attended visit are valid under this assumption, while GEE needs the stronger condition that missingness depends only on the predictors in the model.
Complete-Case Analysis A complete-case analysis keeps only people with no missing values. In repeated-measures data it can discard many people and can bias the results when the people who miss visits differ from those who do not.
Methods & Statistical Concepts
Intracluster Correlation Coefficient (ICC) The intracluster correlation coefficient is the share of the total variation in an outcome that lies between clusters. It equals the between-cluster variance divided by the sum of the between-cluster and within-cluster variances, and it ranges from 0 (no clustering) to 1. The intraclass correlation coefficient, the same kind of variance ratio, is also used for a different purpose: the reliability of a score.
Design Effect The design effect is the factor by which clustering inflates the variance of an estimate compared with a simple random sample of the same size. For clusters of average size m it equals 1 + (m − 1) × ICC.
Effective Sample Size The effective sample size is the number of independent observations that would carry the same information as the clustered sample. It equals the sample size divided by the design effect.
Mixed Model A mixed model is a regression model that contains both fixed effects for the predictors and random effects for the clusters. It is fitted in R with lmer() for a continuous outcome and glmer() for a binary outcome.
Fixed Effect A fixed effect is a coefficient for a predictor in a mixed model, such as the effect of smoking. It is read in the same way as a coefficient of an ordinary regression model.
Random Effect A random effect is a shift for each cluster, up or down from the overall average, that is assumed to come from a normal distribution. The model estimates the variance of these shifts, and ranef() prints the estimated shift for each cluster.
Random Intercept A random intercept gives each cluster its own baseline level of the outcome. In lmer() and glmer() it is written (1 | cluster).
Random Slope A random slope lets the effect of a predictor, such as time, differ from cluster to cluster. It is written (1 + time | cluster), which is the same as (time | cluster), and needs more data than a random intercept, so it often fails to converge in small studies.
Variance Components The variance components of a mixed model are the between-cluster variance and the residual (within-cluster) variance. They are printed under Random effects in the model summary and by VarCorr().
Linear Mixed Model (LMM) A linear mixed model is a linear regression with random effects for the clusters. It is fitted with lmer() from the lme4 package, and loading lmerTest adds p-values for the fixed effects.
Singular Fit A singular fit occurs when a mixed model estimates a variance of zero (or a correlation of exactly ±1) for a random effect. R reports "boundary (singular) fit", and isSingular() returns TRUE. It usually means the random-effects part is more complex than the data can support.
Generalized Linear Mixed Model (GLMM) A generalized linear mixed model is a generalized linear model, such as logistic regression, with random effects for the clusters. It is fitted with glmer(), and its exponentiated fixed effects are cluster-specific odds ratios.
Generalized Estimating Equations (GEE) GEE fits a regression model, such as logistic regression, without cluster effects and calculates standard errors that allow for the correlation within clusters. A working correlation sets how the data are weighted. It is fitted with geeglm() from the geepack package, and its odds ratios are population-averaged.
Working Correlation The working correlation is the pattern of correlation within clusters that a GEE model assumes. The exchangeable structure assumes the same correlation between any two members of a cluster, and an autoregressive structure, AR(1), assumes that measurements closer in time are more alike.
Sandwich Standard Error A sandwich standard error is the standard error that GEE reports. It is calculated by adding up the residuals within each cluster and measuring how much these cluster totals vary from one cluster to another, so it needs a reasonable number of clusters. It remains valid when the working correlation is not exactly right, provided there are enough clusters.
Cluster-Specific (Conditional) Effect A cluster-specific effect compares two people in the same cluster, so the cluster's shift cancels out. For a cluster-level predictor, such as an urban clinic, it compares two clusters with the same baseline. The fixed effects of a mixed model are cluster-specific.
Population-Averaged (Marginal) Effect A population-averaged effect compares the average outcome of two groups across all clusters. GEE estimates population-averaged effects, and for a binary outcome they are usually closer to 1 than the cluster-specific odds ratios of a GLMM.
Group-by-Time Interaction A group-by-time interaction is the term arm:visit, which R includes together with arm and visit when the formula contains arm * visit. It tests whether the outcome changes at a different rate in two groups. In a trial with repeated measures it usually answers the main question.
Key People
Ronald A. Fisher (1890–1962) Fisher was a British statistician and geneticist who described the intraclass correlation, the statistic on which the intracluster correlation coefficient is based, in his 1925 book Statistical Methods for Research Workers.
Leslie Kish (1910–2000) Kish was a Hungarian-born American survey statistician who introduced the design effect in his 1965 book Survey Sampling.
Nan Laird and James Ware Laird and Ware were biostatisticians at Harvard University whose 1982 paper set out the random-effects (mixed) model for longitudinal data in the form that is still used today.
Kung-Yee Liang and Scott Zeger Liang and Zeger were biostatisticians at Johns Hopkins University when they introduced generalized estimating equations for longitudinal data in 1986.
No matching entries. Try a different search term.
Section 1 of 4

Recognising Dependent Data

⏱ Estimated time: 40 minutes
Lesson 5 · Section 1

Recognising Dependent Data

When observations share a clinic, a household or a person, they are no longer independent of one another.

Two forms

Clustered data and repeated measures

Clustered data

People are grouped inside larger units, such as patients in clinics, students in schools or residents in neighbourhoods.

Repeated measures

The same person is measured several times, so each person's measurements form a cluster.

In both forms, observations from the same cluster tend to be more alike than observations from different clusters.

Why it matters

Ignoring dependence makes a model overconfident

  • The model counts every observation as a separate piece of information.
  • Its standard errors are too small, its confidence intervals too narrow and its p-values too small.
  • The problem is largest for predictors that describe the cluster, such as an urban or rural clinic.
Running example

966 patients in 30 clinics

Dot plot of systolic blood pressure for 966 patients, one column per clinic, sorted by clinic average. Clinic averages range from about 102 to 125 mmHg. Urban clinics are teal and rural clinics are orange.
Each column is one clinic and each dot is one patient; the diamonds mark the clinic averages.
Measuring clustering

The intracluster correlation coefficient (ICC)

Intracluster correlation coefficient
\[ \text{ICC} = \frac{\color{#C2410C}{\sigma^2_{\text{between}}}}{\color{#C2410C}{\sigma^2_{\text{between}}} + \color{#0B7B6B}{\sigma^2_{\text{within}}}} \]
σ²between variance between cluster averages σ²within variance among people in the same cluster

The ICC is also the correlation between the outcomes of two people in the same cluster.

What the ICC looks like

Three levels of clustering

Three simulated panels of ten clusters each, with ICC 0, 0.2 and 0.8. With ICC 0 the cluster averages are all near zero; with ICC 0.8 the clusters are tight and far apart.
The panels show simulated clusters with an ICC of 0, 0.2 and 0.8, and the diamonds mark the cluster averages.
ICC in the clinic data

About 17.5% of the variation lies between clinics

22.1
variance between clinics
104.3
variance within clinics
0.175
ICC = 22.1 ÷ (22.1 + 104.3)

ICCs for clinics, schools and neighbourhoods are often between 0.01 and 0.1, so 0.175 indicates strong clustering.

Design effect

From 966 patients to about 150 independent people

Design effect and effective sample size
\[ \text{DEFF} = 1 + (\color{#0B7B6B}{\bar m} - 1) \times \color{#C2410C}{\text{ICC}} \qquad n_{\text{eff}} = \frac{n}{\text{DEFF}} \]
m̄ average cluster size (32.2) ICC intracluster correlation coefficient (0.175)
6.45
design effect ≈ 1 + 31.2 × 0.175
150
effective sample size = 966 ÷ 6.45
What goes wrong

The same difference, with and without allowing for clinics

Two estimates of the urban minus rural difference in systolic blood pressure. Ignoring clinics: 1.49 with 95% CI 0.04 to 2.95. Allowing for clinics: 1.67 with 95% CI minus 2.17 to 5.51.
Ignoring clinics gives p = 0.044; allowing for clinics gives p = 0.40.
Which predictors are affected?

Cluster-level predictors lose the most precision

Cluster-level predictors

These take the same value for everyone in a cluster, such as an urban clinic or a school assigned to one trial arm. Their standard errors grow the most.

Person-level predictors

These vary within each cluster, such as age or smoking. Their standard errors change little and can even shrink.

Two families of methods

Mixed models and GEE

MethodWhat it doesR function
Linear mixed modelGives each cluster its own baseline (a random intercept) for a continuous outcomelme4::lmer()
Generalized linear mixed model (GLMM)Adds a random intercept to a logistic or Poisson modellme4::glmer()
Generalized estimating equations (GEE)Fits a regression for the whole population, with standard errors that allow for clusteringgeepack::geeglm()
Carry forward

What to take into the next section

  • Observations are dependent when they share a cluster, such as a clinic or a person measured several times.
  • The ICC measures how alike observations in a cluster are, and the design effect shows the information lost.
  • Ignoring dependence makes a model overconfident, and mixed models and GEE are the standard remedies.

Overview

Ordinary linear and logistic regression models assume that the observations are independent, so that one person's outcome carries no information about another person's outcome. This lesson deals with data in which that assumption fails, because observations share something, such as patients attending the same clinic or the same person measured at several visits. This section shows how to recognise such dependent data from the design of a study, why ignoring the dependence makes a regression model too confident, why the problem is largest for predictors measured on the cluster, and how the intracluster correlation coefficient (ICC), the design effect and the effective sample size measure the strength of the dependence. The later sections fit models that allow for it.

Learning Objectives

  • 1.1 Identify clustered data and repeated measures from the study design, name the variable that defines the clusters, and explain why ordinary regression output gives no warning of clustering.
  • 2.1 Explain why ignoring dependence makes standard errors, confidence intervals and p-values too small.
  • 2.2 Distinguish cluster-level from person-level predictors and explain why cluster-level predictors are affected most.
  • 2.3 Compare an ordinary regression with a mixed model and describe how the standard error changes.
  • 3.1 Interpret the intracluster correlation coefficient (ICC), the design effect and the effective sample size from R output, and explain why extra clusters add more precision than extra people per cluster.

1.1 Recognising Dependent Data

Data are dependent when observations can be grouped into clusters whose members are more alike than members of different clusters. By this we mean that once you know the outcome of one member of a cluster, you can make a slightly better guess about the outcome of another member of the same cluster, because the two share something that affects the outcome. For example, if one patient at a clinic has high blood pressure, another patient at that clinic is a little more likely than a patient elsewhere to have high blood pressure too, perhaps because the clinic serves an older population or its doctors manage hypertension in a particular way. Two forms of dependence are common in public health. In clustered data, people are grouped inside larger units: patients in clinics, students in schools, residents in neighbourhoods, or workers in workplaces. People in the same cluster share a setting, staff, policies and local conditions, so their outcomes tend to be similar. In repeated measures (also called longitudinal data), the same person is measured more than once, and each person's measurements form a cluster. Clusters can also be nested inside one another, such as patients within doctors within clinics, which this lesson mentions only briefly. Independence, by contrast, is a reasonable assumption when each person was sampled once and separately, as in a random sample drawn from a population register.

Table 5.1 gives four examples of studies that produce dependent data, with the cluster, the observation and the reason why observations in the same cluster are alike. In each case the cluster is the unit whose members tend to be alike: the clinic, the school, the household, or the person in a repeated-measures study, not the visit.

Table 5.1. Examples of clustered data and repeated measures in public health studies.

StudyClusterObservationWhy observations in a cluster are alike
A survey of patients at 30 clinicsClinicPatientPatients share doctors, practices and a local population.
A school-based physical activity programSchoolStudentStudents share teachers, facilities and a neighbourhood.
A household survey of food securityHouseholdHousehold memberMembers share income, meals and housing.
A trial with visits at 0, 6, 12 and 18 monthsPersonVisitEach person brings their own genes, habits and baseline health to every visit.

Three terms recur throughout the lesson. Click each of the cards below to review what a cluster is, what independence means and how it is judged, and what the effective sample size measures; the effective sample size is calculated for the clinic data later in this section.

ClusterClick to explore
IndependenceClick to explore
Effective Sample SizeClick to explore

Dependence is a property of the design, and it has to be recognised from how the data were collected. The caution below states the reason, and it is one of the most common errors in practice.

Clustering Cannot Be Seen in the Regression Output

Nothing in the output of lm() or glm() warns that the observations are clustered. Dependence has to be recognised from the study design and from a variable that identifies the clusters, which is why the first step in any analysis is to ask how the data were collected.

The first three sections of this lesson use phaa_clinics.csv, a simulated dataset of 966 patients attending 30 primary care clinics, with between 18 and 45 patients per clinic (32.2 on average). The patients were sampled through their clinics, so the clinic is the cluster and clinic_id is the variable that identifies it. The file records each patient's age, gender (female = 1 for women), smoking status, BMI, systolic blood pressure (sbp) and whether they were referred to a specialist (referred = 1). Two variables describe the clinic itself: clinic_urban (20 urban and 10 rural clinics) and clinic_size. Figure 5.1 plots the systolic blood pressure of every patient by clinic, so that the differences between clinic averages can be seen before any model is fitted.

Dot plot of systolic blood pressure for 966 patients in 30 clinics, sorted by clinic average. Each column is a clinic; diamonds show clinic averages from about 102 to 125 mmHg. Urban clinics are teal and rural clinics orange.
Figure 5.1. Each column shows the patients of one clinic, sorted from the lowest clinic average (left) to the highest (right), and the diamonds mark the clinic averages. The averages range from about 102 to 125 mmHg, so a patient's blood pressure depends partly on the clinic they attend.

Figure 5.1 shows that a patient’s blood pressure depends partly on the clinic the patient attends, which is exactly the kind of dependence that the design of the study leads us to expect. The next question is what this dependence does to a regression model that ignores it.

2.1 Why Ignoring Dependence Makes Results Look Too Precise

A regression model that assumes independence treats every observation as a separate piece of information. By this we mean that each additional person is assumed to tell the model something new about the outcome, so that 966 patients are treated as 966 distinct pieces of evidence. When observations in a cluster are alike, however, part of what each one says has already been said by the others. For example, once the model has seen ten patients from a clinic whose patients tend to have high blood pressure, an eleventh patient from the same clinic mostly confirms what the first ten already showed about that clinic, rather than adding fully new evidence. As a result, the model overstates how much information it has. Because standard errors are calculated from the amount of information the model believes it has, they come out too small, which in turn makes the confidence intervals too narrow and the p-values too small. The point estimates themselves are often similar to those from a model that allows for clustering; it is the stated certainty around them that is wrong. In practice, this means that an analysis that ignores clustering can report a statistically significant association that the data do not support (Dohoo, Martin & Stryhn, 2012). The problem is therefore one of overconfidence about precision rather than a wrong estimate, and it does not affect every predictor equally, as the next part explains.

2.2 Cluster-Level and Person-Level Predictors

A cluster-level predictor takes one value for everyone in a cluster. Whether a clinic is urban (clinic_urban) is an example: every patient at an urban clinic has the value urban, and every patient at a rural clinic has the value rural. Although the dataset contains 966 rows, the 966 patients therefore supply only 30 independent values of this predictor, one for each clinic. Any comparison of urban with rural is, in effect, a comparison of 20 clinics with 10 clinics, and because the clinics differ from one another for many other reasons, that comparison is far less certain than a comparison of 966 independent people would be. This is why the standard error of a cluster-level predictor is the one most understated when clustering is ignored.

A person-level predictor, by contrast, varies among patients within each clinic. Smoking is an example, because a single clinic has both smokers and non-smokers. This allows the model to compare smokers with non-smokers who attend the same clinic, and in such a comparison the clinic's own baseline is shared by both groups and largely drops out. As a result, the standard error of a person-level predictor changes little when clustering is taken into account (in the Section 2 model, which adds age, gender and smoking, it falls from 0.79 to 0.70, and Section 3 shows a similar pattern for referrals). In fact, as that example shows, allowing for clustering can even make the estimate of a person-level predictor more precise, because once the differences between clinics are set aside, the remaining variation within clinics is smaller and the comparison of people within the same clinic is sharper. A quick test of the level of a predictor is to ask whether two people in the same cluster could have different values of it; if they cannot, the predictor is cluster-level. The comparison that follows shows how large the difference between the two kinds of predictor can be.

2.3 Comparing an Ordinary Regression with a Mixed Model

The question of whether patients in urban clinics have higher blood pressure than patients in rural clinics shows the problem directly, because urban clinic is a cluster-level predictor. An ordinary linear regression with lm() treats the 966 patients as independent. A linear mixed model with lmer() gives each clinic its own baseline, as Section 2 explains. Figure 5.2 compares the two estimates of the urban minus rural difference with their 95% confidence intervals, and Table 5.2 gives the estimates, standard errors and p-values from the two models.

Two estimates of the urban minus rural difference in systolic blood pressure with 95% confidence intervals. Ignoring clinics: 1.49 (0.04 to 2.95). Allowing for clinics: 1.67 (minus 2.17 to 5.51).
Figure 5.2. The two lines show the estimated difference in blood pressure between urban and rural clinics with its 95% confidence interval. The estimates are similar, but the interval from the mixed model is about two and a half times as wide and includes 0.

Table 5.2. The urban minus rural difference in systolic blood pressure from an ordinary regression and a linear mixed model.

ModelUrban minus rural (mmHg)Standard errorp-value
Ordinary regression, lm()1.490.740.044
Mixed model, lmer()1.671.960.40

The ordinary regression suggests a statistically significant difference, and the mixed model shows that the data cannot distinguish urban from rural clinics. The estimate barely changes (1.49 against 1.67 mmHg), but the standard error rises from 0.74 to 1.96 and the p-value from 0.044 to 0.40, because the information about clinic_urban comes from 30 clinics rather than 966 patients. When two outputs like these are compared, the estimate, the standard error and the p-value are read in turn, and the model that allows for clustering is the one to report.

A clustered dataset therefore needs a method that allows for the clustering, and two families of methods are used throughout this lesson. Mixed models add a random effect for each cluster. By this we mean that each clinic is given its own shift up or down from the overall baseline, and rather than estimating 30 separate shifts as fixed numbers, the model treats the shifts as varying around zero and estimates how much they vary. For example, the mixed model in Table 5.2 estimates how far a typical clinic sits above or below the average, and it uses that information to judge how uncertain the urban minus rural difference is. Generalized estimating equations (GEE) take a different route. They fit an ordinary-looking regression with no cluster effects, so the results describe the population as a whole, and then correct the standard errors so that they allow for the correlation within clusters. In other words, a mixed model handles clustering by building the clusters into the model, whereas GEE handles it by adjusting how the uncertainty is calculated. Both need a variable that identifies the clusters. Simpler approaches also exist, such as analysing cluster averages or adding a separate indicator variable for every cluster, but they lose information or cannot estimate the effects of cluster-level predictors, and this lesson does not use them. Table 5.3 lists the three models that the lesson fits, with the outcomes they handle, the R function for each and the sections in which they are used.

Table 5.3. Methods for dependent data used in this lesson.

MethodOutcomeR functionUsed in
Linear mixed modelContinuouslme4::lmer() (with lmerTest for p-values)Sections 2 and 4
Generalized linear mixed model (GLMM)Binary or countlme4::glmer()Section 3
Generalized estimating equations (GEE)Continuous, binary or countgeepack::geeglm()Sections 3 and 4

The mixed model showed that ignoring the clinics understated the standard error for urban clinics by a large factor. The ICC and the design effect, introduced next, predict the size of that factor from the strength of the clustering.

3.1 Measuring Clustering: The ICC, the Design Effect and the Effective Sample Size

Two linked quantities describe how strongly a dataset is clustered. The ICC measures how alike observations in the same cluster are, and the design effect and the effective sample size convert the ICC into a loss of information.

3.1.1 The Intracluster Correlation Coefficient

The intracluster correlation coefficient (ICC) is a measure of how much of the variation in an outcome is due to the clusters. To obtain it, the total variance of the outcome is divided into two parts. The first is a part between clusters, which reflects how much the cluster averages differ from one another. The second is a part within clusters, which reflects how much people in the same cluster differ from one another. The ICC is the share of the total that lies between clusters. By this we mean that if the clinic averages are spread far apart while patients within each clinic are similar, most of the variation is between clinics and the ICC is large; if instead the clinic averages are almost identical and all of the variation comes from differences among patients in the same clinic, the ICC is close to 0. Equation 5.1 expresses this share as a ratio of the two variances.

Intracluster correlation coefficient
\[ \text{ICC} = \frac{\color{#C2410C}{\sigma^2_{\text{between}}}}{\color{#C2410C}{\sigma^2_{\text{between}}} + \color{#0B7B6B}{\sigma^2_{\text{within}}}} \]Eq 5.1
The ICC is the variance between clusters divided by the total of the variance between clusters and the variance within clusters. It runs from 0 (no clustering) to 1 (everyone in a cluster has the same value).

The ICC can also be read as the correlation between the outcomes of two people chosen from the same cluster. These two readings describe the same quantity, and they are linked in a simple way. When a large share of the variation lies between clusters, knowing which cluster a person belongs to tells you a lot about their outcome, and so two people from the same cluster tend to have similar outcomes, which is what a high correlation means. When almost none of the variation lies between clusters, two people from the same cluster are no more alike than two people chosen at random, and the correlation is close to 0. A statistic of the same form, the intraclass correlation coefficient, is also used for a different application: measuring the reliability of a score across occasions or raters. Figure 5.3 shows simulated clusters with three values of the ICC.

Three simulated panels, each with ten clusters of fifteen observations. With ICC 0 the cluster averages are all close to zero. With ICC 0.2 they spread a little. With ICC 0.8 each cluster is tight and the clusters are far apart.
Figure 5.3. The panels show simulated data with ICCs of 0, 0.2 and 0.8, and the diamonds mark the cluster averages. As the ICC rises, the clusters become tighter and their averages move further apart.

Worked Example 5.1 applies Equation 5.1 to the clinic data. It reads the ICC that R reports in Activity 5.1, at the end of this section, and shows how it follows from the two variances; watch that the ICC is the between-clinic share, not the within-clinic share.

Worked Example 5.1: The ICC for Blood Pressure

A mixed model with no predictors (Activity 5.1 below) estimates a between-clinic variance of 22.10 and a within-clinic variance of 104.34. The line icc <- vc$vcov[1] / sum(vc$vcov) in Activity 5.1 divides the first of these by their total, as Equation 5.1 does, and R prints 0.175. About 17.5% of the variation in systolic blood pressure lies between clinics, and the blood pressures of two patients from the same clinic have a correlation of about 0.175.

ICCs for clinics, schools and neighbourhoods in health research are often between 0.01 and 0.1, so a value of 0.175 indicates strong clustering. Small ICCs still matter when clusters are large, as the design effect shows.

The ICC of 0.175 measures the strength of the clustering in the clinic data, and both of its readings, as a share of the variation and as a correlation, describe the same quantity. How much information this clustering removes depends also on the size of the clusters.

3.1.2 The Design Effect and the Effective Sample Size

The design effect (DEFF) expresses the ICC as a loss of information. By this we mean that it tells you how many times larger the variance of an estimate, such as an average or a comparison of cluster characteristics, is in a clustered sample than it would be in a simple random sample of the same number of independent people. A design effect of 1 means that clustering costs nothing, and a design effect of 2 means that the clustered sample carries only about as much information as half as many independent people. The design effect depends on both the ICC and the average cluster size. The ICC matters because it measures how alike people in the same cluster are, and the cluster size matters because the more people are taken from each cluster, the more of them repeat information that the others in their cluster have already supplied. For example, even a modest ICC produces a large design effect when every cluster contains dozens of people. Equation 5.2 gives the design effect and the effective sample size that follows from it.

Design effect and effective sample size
\[ \text{DEFF} = 1 + (\color{#0B7B6B}{\bar m} - 1) \times \color{#C2410C}{\text{ICC}} \qquad n_{\text{eff}} = \frac{n}{\text{DEFF}} \]Eq 5.2
The design effect is 1 plus the average cluster size minus 1, multiplied by the ICC. Dividing the sample size by the design effect gives the effective sample size.

For the clinic data, Activity 5.1 obtains both quantities in R: the line deff <- 1 + (m_bar - 1) * icc applies Equation 5.2 to the average clinic size of 32.2 and the unrounded ICC and prints a design effect of 6.45, and nrow(clinics) / deff prints an effective sample size of about 150 (149.7). For questions about clinic characteristics, the 966 clustered patients therefore carry about as much information as 150 independent people. The formula also shows why cluster size matters: with 32 people per cluster, even an ICC of 0.05 gives a design effect of 2.55, which R returns from 1 + (32 - 1) * 0.05 and which reduces the effective sample size by more than half. At the planning stage the same formula is used to inflate the sample size of a study; this section uses it to describe the information lost in an analysis.

The design effect can also be read directly: it is the factor by which clustering inflates the variance of an estimate about a cluster characteristic, compared with a simple random sample of the same number of people. A design effect of 6.45 means that such an estimate is about 6.45 times as variable as it would be from 966 independent patients, so its standard error is about 2.5 times as large, which is the square root of the design effect and which R returns from sqrt(deff). The comparison in Table 5.2 showed an inflation of this size, with the standard error for urban clinics rising from 0.74 to 1.96. When a study is planned, the same number works in reverse: the sample size needed from a simple random sample is multiplied by the design effect. For example, for a study that recruits 25 people from each cluster and expects an ICC of 0.05, R returns a design effect of 2.2 from 1 + (25 - 1) * 0.05, so a study that needs the precision of 400 independent people needs about 880 people, which R returns from 400 * 2.2, or about 36 clusters of 25 (880 / 25): more than twice as many people as a simple random sample would need. Because the design effect grows with the number of people per cluster, extra people from a cluster that is already in the sample add little new information, and spreading the sample over more clusters usually gains more precision than recruiting more people from each one.

Dependence therefore arises from the design of a study, the ICC and the design effect measure its strength, and ignoring it understates the uncertainty, above all for cluster-level predictors. The narrated R walkthrough below demonstrates all of the methods in Table 5.3 on the lesson’s two datasets, and Activity 5.1 then lets you reproduce the main results of this section, from the ICC and the design effect to the comparison in Table 5.2: run the script alongside the text version and answer the questions from your own output.

Learn to do this in R

Narrated R walkthrough: Mixed Models and GEE in R

This walkthrough uses the same two datasets as this lesson. It measures clustering with the intracluster correlation coefficient, fits and checks a linear mixed model, models a binary outcome with a generalized linear mixed model and with GEE, and analyses repeated measurements over time.

Open the Mixed Models and GEE in R walkthrough
R Activity 5.1: detecting clustering in the clinic data

Text version of the activity: the code of the run-along script, its console output and notes on reading it.

# install.packages("lmerTest")   # run once if needed; it also installs lme4
library(lmerTest)    # loads lme4 (for lmer()) and adds p-values
clinics <- read.csv("https://heal-learn.ca/r-activities/data/phaa_clinics.csv")   # course data, read from the website
clinics$clinic_id <- factor(clinics$clinic_id)   # a label for each clinic
clinics$clinic_urban <- factor(clinics$clinic_urban, levels = c("rural", "urban"))
nlevels(clinics$clinic_id)                     # how many clinics?
summary(as.vector(table(clinics$clinic_id)))   # patients per clinic
Console output
[1] 30 Min. 1st Qu. Median Mean 3rd Qu. Max. 18.00 26.25 32.50 32.20 39.75 45.00

The factor() line tells R to treat clinic_id as a label for each clinic. The file holds 30 clinics with between 18 and 45 patients each (mean 32.2). When lmerTest loads, it prints start-up messages (for example, that it masks lmer() from lme4); they are not errors.

m0 <- lmer(sbp ~ 1 + (1 | clinic_id), data = clinics)
vc <- as.data.frame(VarCorr(m0))
vc[, c("grp", "vcov")]                # variance between and within clinics
icc <- vc$vcov[1] / sum(vc$vcov)      # intracluster correlation coefficient
icc
Console output
grp vcov 1 clinic_id 22.10395 2 Residual 104.34414 [1] 0.1748065

The term (1 | clinic_id) asks for a separate baseline for each clinic. The variance between clinics is 22.10 and the variance within clinics (Residual) is 104.34. The icc line divides the first by the total of the two, and R prints the ICC, 0.175: about 17.5% of the variation in blood pressure lies between clinics.

m_bar <- mean(table(clinics$clinic_id))   # average clinic size
deff <- 1 + (m_bar - 1) * icc             # design effect
deff
nrow(clinics) / deff                      # effective sample size
Console output
[1] 6.453962 [1] 149.6755

R prints a design effect (deff) of 6.45 and an effective sample size of about 150 (149.7): for questions about clinic characteristics, the 966 patients carry about as much information as 150 independent people.

naive <- lm(sbp ~ clinic_urban, data = clinics)                      # ignores clinics
mixed <- lmer(sbp ~ clinic_urban + (1 | clinic_id), data = clinics)   # allows for clinics
summary(naive)$coefficients
summary(mixed)$coefficients
Console output
Estimate Std. Error t value Pr(>|t|) (Intercept) 110.671233 0.5853981 189.052936 0.00000000 clinic_urbanurban 1.493493 0.7421687 2.012336 0.04446231 Estimate Std. Error df t value Pr(>|t|) (Intercept) 110.65727 1.592412 26.91822 69.4903668 6.566440e-32 clinic_urbanurban 1.66819 1.959339 27.40708 0.8514043 4.019229e-01

Reading the output. The first table is from lm(): urban clinics average 1.49 mmHg higher, with a standard error of 0.74 and a p-value of 0.044. The second table is from lmer(): the difference is 1.67 mmHg, but the standard error is 1.96 and the p-value is 0.40. The df column (about 27) reflects the 30 clinics, which are the real units of information for a clinic-level predictor.

Files for this activity Data Code Answer key

R Reflect on what you just ran

Answer each question from your own output, as on the final exam: find the number, say what it means in plain words, and say what it implies for the analysis.

1. Interpret the ICC from m0 (the value of icc) in plain words, in two ways. Is this ICC large by the benchmark taught in the course?

Model answerThe ICC that R prints is 0.175 (the variance between clinics as a share of the total of the two variances), so about 17.5% of the variation in systolic blood pressure lies between clinics (the rest lies between patients within the same clinic). Equivalently, the blood pressures of two patients from the same clinic have a correlation of about 0.175: they tend to be more alike than the blood pressures of two patients from different clinics. Yes, it is large: ICCs for clinics, schools and neighbourhoods are often between 0.01 and 0.1, so 0.175 indicates strong clustering.

2. Interpret the design effect (deff) and the effective sample size. A colleague analysed these 966 patients with ordinary regression and saw no sign of a problem. Why not, and how should clustering be recognised?

Model answerBecause patients in the same clinic are alike, the clustering makes estimates about clinic characteristics about 6.45 times as variable (design effect 6.45) as they would be from 966 independent patients. For questions about clinic characteristics, the 966 patients carry about as much information as 150 independent patients (the effective sample size, which R prints as 149.7), so analyses that ignore the clustering overstate precision. Nothing in the output of lm() or glm() shows that observations are clustered, so the colleague's clean output is no evidence that the patients are independent. Clustering is recognised from the study design (patients sampled through clinics) and from a variable that identifies the clusters (clinic_id).

3. Run your own code: at the end of the script, type confint(mixed, parm = "beta_", method = "Wald") and run it. Compare the standard error and p-value for clinic_urbanurban in the two models, and explain why they change so much. Interpret the urban-clinic coefficient from the mixed model, with its 95% confidence interval. Which result should be reported, and why?

Model answerThe standard error for urban clinics is much larger in the mixed model (1.96 against 0.74), and the p-value rises from 0.044 to 0.40, while the estimate barely changes (1.49 against 1.67 mmHg). Urban or rural is a characteristic of the clinic: it takes one value per clinic, so the 966 patients supply only 30 independent values of it (20 urban and 10 rural clinics). Ordinary regression treats every patient as independent, so for a clinic-level predictor its standard error, confidence interval and p-value are too small (overconfident). Patients of urban clinics average about 1.7 mmHg higher systolic blood pressure than patients of rural clinics (1.67 mmHg), but the 95% CI from your confint() line, −2.17 to 5.51, includes 0, so the data cannot tell urban and rural clinics apart; the comparison is also observational. Report the mixed model, because it allows for patients in the same clinic being alike; the ordinary regression overstates the evidence.
Saved.

Your output from Activity 5.1 confirms that the 30 clinics, not the 966 patients, carry the information about clinic characteristics. The key takeaways below draw the section together before its knowledge check.

Key Takeaways

  • 1.1 Dependent data come in two common forms, clustered data and repeated measures, and you recognise both from how the study was designed rather than from the numbers in the dataset. Observations are dependent when they share something: patients in the same clinic share doctors and a local population, students in a school share teachers and facilities, and the visits of one person share that person's habits and baseline health. When you read a question, ask how people were sampled and measured; if they were recruited in groups or measured more than once, the data are clustered, and the cluster is the unit whose members tend to be alike (the clinic, the school, or the person in a repeated-measures study, not the visit or the year). Then name the variable that identifies the clusters, such as clinic_id or a participant ID, because every method in this lesson needs it. Do not wait for R to warn you: nothing in the output of lm() or glm() shows that observations are clustered, so a clean regression table is no evidence that the data are independent.
  • 2.1 Ignoring clustering usually leaves the estimates much the same, but it makes the model far too sure of them. A model that assumes independence counts every observation as a separate piece of information, yet people in the same cluster partly repeat what the others have already said, so the model believes it has more information than it really does. As a result the standard errors are too small, the confidence intervals too narrow and the p-values too small, and an analysis can report a statistically significant association that the data do not support. When a question asks what goes wrong, name all three quantities and the direction in which each is wrong, and describe the problem as overconfidence about precision rather than as a wrong estimate.
  • 2.2 Before you interpret any standard error from clustered data, decide whether each predictor describes the cluster or the person. A cluster-level predictor, such as an urban clinic or a school-wide program, takes one value for everyone in a cluster, so 966 patients in 30 clinics supply only 30 independent values of it, and its standard error is the one most understated when clustering is ignored. A person-level predictor, such as smoking or age, varies among people within each cluster, so the model can compare people in the same clinic, and allowing for clustering changes its standard error little or can even make it smaller (for smoking it fell from 0.79 to 0.70 in the Section 2 model). A quick test is to ask whether two people in the same cluster could have different values: if they cannot, the predictor is cluster-level. In an exam answer, expect and explain a large rise in the standard error and p-value for a cluster-level predictor and little change for a person-level one.
  • 2.3 Putting an ordinary regression and a mixed model side by side is the clearest way to see what clustering does. In the clinic data, the urban minus rural difference in blood pressure was similar in both models (1.49 mmHg from lm() and 1.67 mmHg from lmer()), but the standard error rose from 0.74 to 1.96 and the p-value from 0.044 to 0.40, so an apparently significant result disappeared once the 30 clinics were treated as the real units of information. When a question gives you two outputs like these, compare the estimate, the standard error and the p-value in turn, say that you would report the model that allows for clustering, and explain the change by pointing out that the predictor is measured on the cluster. Mixed models and GEE are the two standard remedies, and the rest of the lesson shows you how to fit and read them.
  • 3.1 The ICC tells you how alike people in the same cluster are, and you can read it in two equivalent ways: as the share of the outcome's variation that lies between clusters, or as the correlation between the outcomes of two people from the same cluster. R obtains it from the two variances of a mixed model with no predictors: the line vc$vcov[1] / sum(vc$vcov) gives the between-cluster share of the total, between ÷ (between + within), and prints 0.175 for blood pressure in the clinics; a common slip in reading it is to describe the within-cluster share instead. Judge its size against the usual range of 0.01 to 0.1 for clinics, schools and neighbourhoods, but remember that a small ICC still matters when clusters are large. The design effect, 1 + (average cluster size − 1) × ICC, is the factor by which clustering inflates the variance of an estimate about a cluster characteristic (R printed 6.45 here), and the effective sample size, n ÷ DEFF, is the number of independent people who would carry the same information (about 150 here), so keep the phrase "for questions about cluster characteristics" in your interpretation. Because the design effect grows with the number of people per cluster, a study gains more precision from extra clusters than from extra people in the clusters it already has, and a planned clustered study needs the sample size of a simple random sample multiplied by the design effect.
Knowledge check: this section

1. A health unit surveys 840 adults living in 40 social-housing apartment buildings (about 21 per building) about respiratory symptoms, and the data file has a column building_id. A colleague fits lm() and says: “The output showed no warning, so the 840 adults can be treated as independent.” Which response is correct?

lm() never checks whether residuals from the same group are correlated, so the absence of a warning says nothing about independence. Clustering is recognised from the study design and from a variable such as building_id, not from the output.
Dependence is a property of how the data were collected: adults sampled in groups from buildings that share conditions affecting symptoms have alike outcomes, and building_id identifies the clusters the analysis must allow for.
R² describes how much variation the predictors explain, not whether observations are correlated within groups. Nothing in ordinary regression output, R² included, signals clustering; the sampling design does.
This confuses being measured once with being independent. Adults sampled in groups that share a building's air, neighbours and management tend to have similar outcomes, so they are clustered even with one interview each.
Correct answer: ordinary regression output cannot show clustering, and the building is the cluster. The information needed is how the adults were sampled: in groups of about 21 from 40 buildings, with building_id recorded. Nothing in the output of lm() or glm() (coefficients, R² or residuals) signals clustering, so dependence is recognised from the study design and from the variable that identifies the clusters; adults who share a building share conditions that affect their symptoms, so their outcomes are alike. The tempting answer that each adult was interviewed only once confuses being measured once with being independent: people sampled in groups that share a setting are not independent. (Section 1, 1.1 Recognising Dependent Data and the callout “Clustering Cannot Be Seen in the Regression Output”; Objective 1.1)

2. In the housing survey (840 adults in 40 buildings), a mixed model with no predictors, m0 <- lmer(symptoms ~ 1 + (1 | building_id), data = housing), was fitted and its ICC obtained in R as shown below. What does this ICC mean?

> vc <- as.data.frame(VarCorr(m0)) > vc[, c("grp", "vcov")] grp vcov 1 building_id 3.012448 2 Residual 16.953871 > icc <- vc$vcov[1] / sum(vc$vcov) > icc [1] 0.1508765
The ICC is a share of the variation in the outcome, not a share of the people. Every adult belongs to a building, so clustering applies to the whole sample rather than to 15% of it.
This reverses the two shares. The ICC is the building variance as a share of the total, so 0.15 is the between-building share; the remaining 85% is variation among adults within buildings.
The ICC has two readings: the share of total variation that lies between clusters and the correlation between two members of the same cluster. Compared with the usual 0.01 to 0.1 for such settings, 0.15 is strong clustering.
An ICC of 0.15 is above the usual range, and even a smaller ICC matters when clusters are large, because the design effect depends on cluster size as well as the ICC. The ICC alone is never a reason to ignore clustering.
Correct answer: about 15% of the variation lies between buildings, and the scores of two adults in the same building are correlated at about 0.15. The information needed is the value that R printed for icc, 0.15, which is the building variance (the building_id row, 3.01) as a share of the total of the two variances (Equation 5.1), and the benchmark that ICCs for clinics, schools and neighbourhoods are often between 0.01 and 0.1, so 0.15 indicates strong clustering. The ICC is a share of the variation, not a share of the people, and the within-building share (about 85%) is the remainder, not the ICC. Even a small ICC matters when clusters are large, as the design effect in Question 3 shows, so an ICC is not on its own a reason to ignore clustering. (Section 1, 3.1.1 The Intracluster Correlation Coefficient, Equation 5.1 and Worked Example 5.1; Objective 3.1)

3. The analyst continues in R with the ICC from Question 2 and the average building size of 21 adults, as shown below. Which statement reads the two results correctly?

> m_bar <- mean(table(housing$building_id)) # 21 adults per building > deff <- 1 + (m_bar - 1) * icc # design effect > deff [1] 4.01753 > nrow(housing) / deff # effective sample size [1] 209.0837
This overgeneralizes the loss of information. The design effect applies to estimates about building characteristics; a person-level predictor such as smoking varies within buildings and can be compared there, so its precision changes little.
This reverses the direction. The design effect is the factor by which clustering inflates the variance of a cluster-level estimate, so clustering reduces the information available; it never adds information.
The number is right but the reading is wrong. The effective sample size describes how much information all 840 adults carry together, not a count of particular independent people, so no one is removed.
The design effect inflates the variance of an estimate about a cluster characteristic, and the effective sample size is the number of independent people carrying the same information, so the loss is stated for building-level questions.
Correct answer: a design effect of about 4.0 and an effective sample size of about 209, both for questions about building characteristics. The information needed is what each printed number means. The design effect (Equation 5.2) is the factor by which clustering inflates the variance of an estimate about a cluster characteristic, and the effective sample size, n ÷ DEFF, is the number of independent people who would carry the same information. The option that removes 631 adults has the right number but the wrong reading, because no one is dropped: the 840 adults together carry the information of about 209 independent adults. The loss applies to questions about building characteristics; a person-level predictor such as smoking can be compared within buildings, so its precision changes little, and clustering never adds information. (Section 1, 3.1.2 The Design Effect and the Effective Sample Size and Equation 5.2; Objective 3.1)

4. In the housing survey, 20 of the 40 buildings have a smoke-free policy. The symptom score was modelled with lm(), which ignores the buildings, and with lmer() with a random intercept for each building, giving the results below for the policy. Which result should be reported, and why?

ModelEstimateStandard errorp-value
lm()−1.100.30< 0.001
lmer()−1.200.620.06
The policy is a cluster-level predictor, so its information comes from the 40 buildings. A model that treats the 840 adults as independent overstates that information, making its standard error, interval and p-value too small.
The smaller standard error does not come from extra information but from the false assumption that the adults are independent. For a building-level predictor, lm() is overconfident rather than more precise.
The estimates are similar, but the standard error and p-value are not: allowing for clustering roughly doubles the standard error and moves the p-value above 0.05. Ignoring clustering changed the strength of evidence, not the estimate.
The right model is chosen for the wrong reason. Ignoring clustering mainly makes results look too precise; the two estimates are nearly the same, so the mixed model is preferred for its honest standard error, not for correcting a bias.
Correct answer: report the lmer() result. The information needed is that the policy is a building-level predictor and that, between the two models, the estimate barely changes (−1.10 against −1.20) while the standard error doubles (0.30 to 0.62, about √4.0 = 2 times, as the design effect of Question 3 suggests) and the p-value rises from below 0.001 to 0.06. A model that treats clustered observations as independent overstates how much information it has, so for a cluster-level predictor its standard error, confidence interval and p-value are too small; the mixed model shows that 40 buildings give only weak evidence about the policy. The smaller standard error of lm() is tempting, but it comes from a false assumption of independence, not from extra information. (Section 1, 2.1 Why Ignoring Dependence Makes Results Look Too Precise and 2.3 Comparing an Ordinary Regression with a Mixed Model, Table 5.2; Objectives 2.1 and 2.3)

5. The same two models also include two predictors measured on each adult, with the standard errors shown below. Why does the standard error for the smoke-free policy double, while those for current smoking and age barely change?

PredictorSE in lm()SE in lmer()
Smoke-free policy (building)0.300.62
Current smoker (adult)0.260.24
Age, per 10 years (adult)0.0900.088
The size of an effect does not determine its standard error. What matters is the level at which the predictor varies: the policy takes one value per building, so its information is limited to the 40 buildings.
A cluster-level predictor has only as many independent values as there are clusters, while person-level predictors vary within clusters and can be compared between adults in the same building, so their standard errors barely change.
The table rules this out: the standard errors for smoking and age fall slightly in the mixed model. The mixed model is not less precise for every predictor; the correction matters most for predictors that do not vary within clusters.
The mixed model allows for clustering for every predictor. Smoking and age change little because they vary within buildings and can be compared within them, not because they are left out of the correction.
Correct answer: the policy is a building-level (cluster-level) predictor, while smoking and age are person-level predictors. The information needed is the level at which each predictor varies: a cluster-level predictor takes one value per cluster, so the data hold only as many independent values of it as there are clusters (40), whereas a person-level predictor varies within clusters and can be compared between adults in the same building. The table shows the standard errors for smoking and age falling slightly (0.26 to 0.24 and 0.090 to 0.088), which rules out the claim that the mixed model is less precise for every predictor. The mixed model does allow for clustering for every predictor; the correction simply matters most for predictors that do not vary within clusters. (Section 1, 2.2 Cluster-Level and Person-Level Predictors; Objective 2.2)

6. A health authority surveyed 30 patients in each of 32 emergency departments, and the ICC for a patient-experience score is 0.10. For next year, a colleague says: “Surveying 40 patients in each of the same 32 departments gains just as much precision as adding 10 more departments of 30 patients.” The effective sample size of each plan, the number of patients divided by its design effect, was obtained in R as shown below. Which response is correct?

> 32 * 40 / (1 + (40 - 1) * 0.10) # 40 patients in each of the same 32 departments [1] 261.2245 > 42 * 30 / (1 + (30 - 1) * 0.10) # 42 departments of 30 patients [1] 323.0769
With clustered data, equal numbers of patients do not mean equal precision. The plan with larger clusters has a larger design effect, so its effective sample size is clearly smaller despite a similar total.
This reverses the lesson's point. The design effect grows with the number of people per cluster, so adding patients to departments already sampled adds less new information than adding departments.
Because the design effect grows with cluster size, extra patients from the same departments partly repeat information already collected, while each new department contributes independent information, giving the larger effective sample size.
The ICC is the same in both plans, but the design effect also depends on the number of people per cluster. The plans have different cluster sizes, so their design effects and effective sample sizes differ.
Correct answer: adding departments gives more information (about 323 against about 261 independent patients). The information needed is the two effective sample sizes that R printed and what drives them: the design effect, 1 + (average cluster size − 1) × ICC, grows with the number of people per cluster, so 40 patients per department give a larger design effect than 30, and each added department brings new, independent information. Equal numbers of patients do not mean equal precision when the data are clustered, which is why the colleague is wrong. The ICC is the same in both plans, but the design effect is not, because it also depends on cluster size. (Section 1, 3.1.2 The Design Effect and the Effective Sample Size and Equation 5.2; Objective 3.1)
● Complete the knowledge check to continue.
Section 2 of 4

Linear Mixed Models

⏱ Estimated time: 40 minutes
Lesson 5 · Section 2

Linear Mixed Models

A model for a continuous outcome that gives each cluster its own baseline.

The idea

Each clinic gets its own baseline

Random-intercept linear mixed model
\[ \text{SBP}_{ij} = \color{#6D28D9}{\beta_0} + \color{#C2410C}{u_j} + \color{#1D4ED8}{\beta_1}\,\text{age}_{ij} + \dots + \color{#0B7B6B}{\varepsilon_{ij}}, \qquad \color{#C2410C}{u_j} \sim N(0, \sigma^2_u) \]
β0 overall intercept uj shift for clinic j (random effect) β1, … fixed effects, read as in Lesson 4 εij difference for patient i within the clinic

The model estimates one variance, σ²u, that describes how much the clinic baselines differ.

Two kinds of effects

Fixed effects and random effects

Fixed effects

These are the coefficients for the predictors, read as differences in average blood pressure, as in Lesson 4.

Random effects

These are the clinic shifts, summarised by their variance or standard deviation.

lmer(sbp ~ age + female + smoker + clinic_urban + (1 | clinic_id), data = clinics)

The term (1 | clinic_id) asks for a random intercept for each clinic.

Reading the output

Fixed effects from lmer()

PredictorEstimate (mmHg)95% CIp-value
Age (per year)0.410.37 to 0.45< 0.001
Female−2.80−3.88 to −1.73< 0.001
Smoker5.003.63 to 6.37< 0.001
Urban clinic1.85−1.85 to 5.540.34

The random effects show a standard deviation of 4.6 mmHg between clinic baselines and 8.3 mmHg among patients within a clinic.

The clinic effects

Clinics differ by several mmHg, after adjustment

Estimated clinic effects for 30 clinics, sorted from lowest to highest, with 95% intervals. Most lie within 5 mmHg of zero; three clinics lie about 8 to 11 mmHg above average.
The adjusted ICC is 21.6 ÷ (21.6 + 68.9) = 0.24, and each point is one clinic's estimated shift from the overall average.
Interpretation

What a fixed effect means

  • The smoking coefficient compares a smoker and a non-smoker of the same age and gender in the same clinic.
  • For a continuous outcome, this is also the average difference in the whole population.
  • The main gain from the mixed model is a standard error that reflects the clustering.
Model checking

Three plots check most of the assumptions

Three diagnostic plots for the linear mixed model: residuals versus fitted values with a flat smoother, a Q-Q plot of residuals close to the line, and a Q-Q plot of the 30 clinic effects with three high points above the line.
Panels A and B look acceptable; panel C shows three clinics with higher baselines than a normal distribution predicts.
Further checks

Enough clusters, and no singular fit

AssumptionHow to check itThis example
Linearity and equal varianceResiduals versus fitted valuesAcceptable
Normal residualsQ-Q plot of residualsAcceptable
Normal clinic effectsQ-Q plot of the random effectsThree high clinics
Enough clusters (about 20 to 30)nlevels(clinic_id)30 clinics
Clinic variance estimatedisSingular(model)FALSE
Going further

Random slopes

Random intercept

Clinic baselines differ, and each predictor has the same effect in every clinic.

(1 | clinic_id)

Random slope

The effect of a predictor, such as age, is also allowed to differ between clinics.

(age | clinic_id)

Random slopes need more clusters and often trigger warnings with small samples.

Carry forward

What to take into the next section

  • A random intercept gives each cluster its own baseline, summarised by one variance.
  • Fixed effects are read as in an ordinary regression, and random effects describe the clusters.
  • The checks are residual plots, a Q-Q plot of cluster effects, the number of clusters and the singular-fit warning.

Overview

This section presents the linear mixed model, the standard model for a continuous outcome when observations are clustered. It continues with systolic blood pressure in the 30 clinics of phaa_clinics.csv, now with four predictors: age, gender, smoking and whether the clinic is urban. It explains what a random intercept adds to an ordinary linear regression, how to read the fixed effects, the estimated clinic shifts and the variance components from the R output, which checks a mixed model needs, and what a random slope adds and why it often cannot be estimated.

Learning Objectives

  • 4.1 Describe a random intercept and explain how a linear mixed model differs from an ordinary linear regression.
  • 4.2 Interpret the fixed effects of a random-intercept model fitted with lmer(), and the estimated shift of a single cluster.
  • 4.3 Interpret the random-effect variances and the adjusted ICC from R output.
  • 5.1 Check a linear mixed model with residual plots, a Q-Q plot of the cluster effects, the number of clusters and the singular-fit check.
  • 5.2 Explain what a random slope adds, when it is used, and what a singular-fit or convergence warning means for the choice of model.

4.1 The Random-Intercept Model

An ordinary linear regression has a single intercept and treats every patient as independent. In other words, it assumes that every clinic starts from the same baseline blood pressure, and that any difference between two patients with the same predictor values is purely a matter of individual variation. A random-intercept model relaxes this assumption by giving each cluster its own intercept. By this we mean that each clinic's baseline is made up of two parts: the overall intercept, which is shared by all clinics, plus a cluster-specific shift, which moves that clinic's baseline up or down. For example, a clinic that serves an older or less healthy population might have a shift of several mmHg above the average, while another clinic might sit a few mmHg below it.

What makes the intercept "random" is the way these shifts are treated. Rather than estimating a separate coefficient for each of the 30 clinics, the model treats the shifts as a random sample from a normal distribution with mean 0, and it estimates the variance of that distribution, which describes how much the clusters differ (Laird & Ware, 1982). A small variance means that the clinic baselines are close together, and a large variance means that they are spread far apart. Equation 5.3 writes the model for systolic blood pressure in the clinic data.

Random-intercept model for systolic blood pressure
\[ \text{SBP}_{ij} = \color{#6D28D9}{\beta_0} + \color{#C2410C}{u_j} + \color{#1D4ED8}{\beta_1}\,\text{age}_{ij} + \color{#1D4ED8}{\beta_2}\,\text{female}_{ij} + \color{#1D4ED8}{\beta_3}\,\text{smoker}_{ij} + \color{#1D4ED8}{\beta_4}\,\text{urban}_{j} + \color{#0B7B6B}{\varepsilon_{ij}} \]Eq 5.3
The blood pressure of patient i in clinic j equals the overall intercept, plus a shift for clinic j, plus the fixed effects of the predictors, plus a difference for the patient within the clinic. The clinic shifts uj have variance σ²u, and the patient differences εij have variance σ²e. The subscript j on urban shows that it is a clinic-level predictor.

The name "mixed" refers to the two kinds of effects in the model. The fixed effects (β0 to β4) are the coefficients for the predictors, read as differences in average blood pressure, as in ordinary linear regression. The random effects (uj) are the clinic shifts, summarised by their variance. This way of handling the clinics has an important advantage. If the model instead included 30 separate clinic coefficients, those coefficients would absorb every difference between clinics, including the difference between urban and rural clinics, and the model could no longer estimate the effect of a clinic-level predictor at all. Summarising the 30 clinics with one variance avoids this problem. Because the clinic shifts are described by their spread rather than fixed one by one, the model can allow for the clustering and still estimate the effect of a clinic-level predictor such as clinic_urban, with a standard error that reflects the 30 clinics. Click each of the three cards below to review the random intercept, with its lmer() notation (1 | clinic_id), the fixed effects and the variance components.

Random InterceptClick to explore
Fixed EffectsClick to explore
Variance ComponentsClick to explore

The random intercept therefore gives each clinic its own baseline while the fixed effects describe the predictors. With the parts of the model defined, the next part reads the fixed effects from the R output for the clinic data.

4.2 Interpreting the Fixed Effects and Cluster Shifts

Activity 5.2 at the end of this section fits the model with lmer() after loading the lmerTest package, which adds p-values to the fixed effects. The output has two main blocks, the fixed effects and the random effects. Table 5.4 summarises the fixed-effects block, with a 95% confidence interval and an interpretation for each predictor.

Table 5.4. Fixed effects from the random-intercept model for systolic blood pressure.

Fixed effectEstimate (mmHg)95% CIp-valueInterpretation
Age (per year)0.410.37 to 0.45< 0.001Each extra year of age is associated with 0.41 mmHg higher blood pressure.
Female−2.80−3.88 to −1.73< 0.001Women average 2.8 mmHg lower than men.
Smoker5.003.63 to 6.37< 0.001Smokers average 5.0 mmHg higher than non-smokers.
Urban clinic1.85−1.85 to 5.540.34There is no clear evidence of a difference between urban and rural clinics.

Each estimate is read like a coefficient of an ordinary linear regression and holds the other predictors constant, so it compares patients with the same values of the other predictors in the same clinic: smokers, for example, average 5.0 mmHg higher than non-smokers of the same age and gender attending the same clinic. Because the clinics were not assigned to be urban or rural and the patients were not assigned to smoke, these fixed effects describe associations, not the effects of changing a predictor. The df column of the output shows about 937 degrees of freedom for the patient-level predictors and about 28 for urban clinics. The degrees of freedom reflect how much independent information stands behind each estimate. For age, gender and smoking, the information comes from comparisons among the 966 patients, so the degrees of freedom are large. For urban clinic, by contrast, the information comes from the 30 clinics, so the degrees of freedom are close to the number of clinics, and the test for this predictor is correspondingly less certain. The confidence interval for urban clinic includes 0, so the data cannot tell urban and rural clinics apart.

The model also estimates a shift for each clinic. Figure 5.4 shows the estimated shifts of the 30 clinic baselines from the overall average.

Estimated clinic effects for 30 clinics with 95% intervals, sorted from lowest (about minus 8 mmHg) to highest (about plus 11 mmHg). Most lie within 5 mmHg of zero.
Figure 5.4. Each point is the estimated shift of one clinic's baseline from the overall average, after adjusting for the predictors, with a 95% interval. Most clinics lie within 5 mmHg of the average, and three lie about 8 to 11 mmHg above it.

An estimated shift is read for the whole clinic: patients of a clinic with a shift of +8 mmHg have blood pressures about 8 mmHg higher, on average, than patients with the same age, gender, smoking status and clinic type in an average clinic. The shift describes a difference that the predictors in the model do not explain, such as local practices or the population a clinic serves, not the effect of any measured variable. Worked Example 5.2 puts the fixed effects and a clinic shift together to predict blood pressure for two patients; watch what the shift does to each prediction and to the difference between them.

Worked Example 5.2: Predicted Blood Pressure for Two Patients

For a 50-year-old male non-smoker in a rural clinic, predict() in R gives 110.4 mmHg in an average clinic, and for a 50-year-old male smoker in the same clinic it gives 115.4 mmHg. The two patients differ only in smoking, so the gap between the predictions is the smoking coefficient of 5.0 mmHg. In a clinic whose estimated shift is +8 mmHg, both predictions rise by 8 mmHg, but the difference between them stays 5.0 mmHg, because the clinic shift is shared by everyone in the clinic.

The worked example shows that a fixed effect compares patients within a clinic and a clinic shift moves the whole clinic. The spread of the shifts in Figure 5.4 is summarised by the clinic variance, which the next part reads from the random-effects block.

4.3 The Variance Components and the Adjusted ICC

The random effects block reports a clinic variance of 21.57 (standard deviation 4.64 mmHg) and a residual variance of 68.94 (standard deviation 8.30 mmHg). The standard deviations are in the units of the outcome, which makes them easier to picture. For example, a clinic standard deviation of 4.64 mmHg means that a typical clinic's baseline sits roughly 4 or 5 mmHg above or below the average clinic, after allowing for the predictors, while a residual standard deviation of 8.30 mmHg means that patients within the same clinic typically differ from their clinic's expected value by around 8 mmHg. The adjusted ICC, which the line vc$vcov[1] / sum(vc$vcov) in Activity 5.2 prints as 0.24, is the clinic share of the total of these two variances. It is larger than the ICC of 0.175 from the model with no predictors, because age, gender and smoking explain much of the variation among patients within clinics (the residual variance falls from 104.34 to 68.94) but little of the variation between clinics (about 22 in both models). Because the between-clinic part stayed about the same while the within-clinic part shrank, the between-clinic part became a larger share of what was left, and so the ICC rose. The adjusted ICC is not always larger, however. Which way it moves depends on which kind of predictor is added. Person-level predictors mainly reduce the residual variance, because they explain why patients in the same clinic differ from one another. A cluster-level predictor that is strongly related to the outcome, by contrast, explains part of the differences between clusters and so reduces the cluster variance. If the cluster variance falls by a larger share than the residual variance, the adjusted ICC is smaller than the ICC from the model with no predictors. Any change in the ICC is therefore explained by looking at which of the two variances fell, and by how much.

These interpretations depend on the model being appropriate for the data, so the next part sets out the checks that a linear mixed model needs.

5.1 Checking a Linear Mixed Model

A linear mixed model makes the assumptions of linear regression, applied to the residuals within clusters, and adds assumptions about the clusters. Table 5.5 lists each assumption with what it means and how to check it, and Figure 5.5 shows the three diagnostic plots for the blood pressure model.

Table 5.5. Assumptions of a linear mixed model and how to check them.

AssumptionWhat it meansHow to check it
Linearity and equal varianceEach continuous predictor has a straight-line relationship with the outcome, and the residuals have the same spread at every fitted value.Plot of residuals against fitted values.
Normal residualsThe differences among patients within a clinic are roughly normal.Q-Q plot of the residuals.
Normal cluster effectsThe clinic shifts come from a roughly normal distribution.Q-Q plot of ranef(model). With few clusters this check is rough.
Enough clustersThe cluster variance is estimated from the clusters, so a reasonable number is needed.nlevels(clinic_id); about 20 to 30 clusters is a common minimum.
Independent clustersClinics are unrelated to one another, and patients are independent within a clinic once the clinic effect is allowed for.Judged from the study design.
Cluster variance estimatedThe model is not singular, so the cluster variance is not estimated as zero.isSingular(model) returns FALSE; a "boundary (singular) fit" message signals a problem.
Three diagnostic plots for the linear mixed model. A: residuals versus fitted values form a flat, even cloud. B: residuals follow the Q-Q line closely. C: the 30 clinic effects follow the Q-Q line except for three high points.
Figure 5.5. Panel A supports linearity and equal variance, and panel B supports normal residuals. Panel C shows that three clinics have higher baselines than a normal distribution would predict, which is worth reporting and investigating.

In this example the residual checks look acceptable, there are 30 clinics, and the model is not singular, because isSingular() returns FALSE, which means the clinic variance was estimated rather than collapsing to zero. Passing the singular-fit check does not show that there are enough clusters, which is a separate check. With fewer than about 40 clusters, a small-sample correction to the tests is advised (Hemming & Taljaard, 2023), and the lmerTest output in this section already uses one (Satterthwaite's method). The three high clinics in panel C are a mild departure from normal cluster effects. Fixed-effect estimates are usually not very sensitive to this assumption, so such clinics are reported and investigated rather than treated as a reason to abandon the model, for example to see whether they serve an older population or record blood pressure differently.

The checks support the random-intercept model for the clinic data. The final part of this section considers a more flexible model, in which the effect of a predictor may differ between clinics.

5.2 Random Slopes and Model Warnings

The random-intercept model lets clinic baselines differ but assumes that each predictor has the same effect in every clinic. A random slope lets the effect of a predictor vary between clusters as well. By this we mean that, in addition to each clinic having its own baseline, each clinic can also have its own version of the relationship between a predictor and the outcome. For example, blood pressure might rise by 0.41 mmHg per year of age on average, but more steeply in one clinic and less steeply in another. In lmer(), a random slope for age is written (age | clinic_id), which gives each clinic its own intercept and its own age slope. Just as with the random intercept, the clinic-specific slopes are not estimated one by one; the model estimates how much the slopes vary around the average slope. Random slopes suit research questions about how an effect differs between settings, such as whether a program works better in some schools than others. This added flexibility comes at a cost, however. Measuring how much a slope varies between clusters is much harder than measuring how much a baseline varies, so random slopes need more clusters and more observations per cluster, and with 30 clinics they often produce a singular fit or a convergence warning. This lesson fits random intercepts only.

Both warnings say that the random part of the model is more complex than the data can support. A convergence warning means that R stopped before it was sure it had found the best estimates, and a singular fit means that a variance was estimated at or near zero (or a correlation between the random intercepts and slopes at exactly ±1). It is tempting to read a singular fit for a random slope as evidence that the effect is the same in every clinic, but this is a mistake. Neither warning shows that the effect is the same in every cluster; it shows that the data cannot measure how much the effect varies. In other words, the problem is a lack of information, not a finding about the clinics. The usual response is to return to the simpler random-intercept model, report it, and state that a random slope could not be estimated. If the main fixed effect is almost the same in the two models, the conclusion does not depend on the random slope.

The linear mixed model with a random intercept for each clinic, read through its fixed effects, clinic shifts, variance components and adjusted ICC, and checked against Table 5.5, is the basic model for clustered continuous outcomes. Activity 5.2 lets you fit it in R, run the checks and try a random slope for age: run the script and answer the questions from your own output. Section 3 then applies the same approach to a binary outcome.

R Activity 5.2: a random-intercept model for blood pressure

Text version of the activity: the code of the run-along script, its console output and notes on reading it.

library(lmerTest)   # loads lme4 and adds p-values to lmer() output
clinics <- read.csv("https://heal-learn.ca/r-activities/data/phaa_clinics.csv")   # course data, read from the website
clinics$clinic_id <- factor(clinics$clinic_id)
clinics$smoker <- factor(clinics$smoker, levels = c("No", "Yes"))
clinics$clinic_urban <- factor(clinics$clinic_urban, levels = c("rural", "urban"))
nlevels(clinics$clinic_id)   # how many clinics?
Console output
Loading required package: lme4 Loading required package: Matrix Attaching package: 'lmerTest' The following object is masked from 'package:lme4': lmer The following object is masked from 'package:stats': step [1] 30

R prints messages as it loads lmerTest and lme4; they are not errors. nlevels() confirms 30 clinics, which meets the usual minimum of about 20 to 30 clusters for a mixed model.

lmm <- lmer(sbp ~ age + female + smoker + clinic_urban + (1 | clinic_id),
            data = clinics)
summary(lmm)
confint(lmm, parm = "beta_", method = "Wald")   # 95% CIs for the fixed effects
head(ranef(lmm)$clinic_id, 3)   # estimated shifts of clinics 1 to 3 (mmHg)
Console output
Linear mixed model fit by REML. t-tests use Satterthwaite's method [ lmerModLmerTest] Formula: sbp ~ age + female + smoker + clinic_urban + (1 | clinic_id) Data: clinics REML criterion at convergence: 6896.6 Scaled residuals: Min 1Q Median 3Q Max -2.95086 -0.65976 -0.02077 0.66101 2.98357 Random effects: Groups Name Variance Std.Dev. clinic_id (Intercept) 21.57 4.644 Residual 68.94 8.303 Number of obs: 966, groups: clinic_id, 30 Fixed effects: Estimate Std. Error df t value Pr(>|t|) (Intercept) 90.0114 1.8891 62.2481 47.647 < 2e-16 *** age 0.4076 0.0201 937.0898 20.282 < 2e-16 *** female -2.8050 0.5465 937.4243 -5.133 3.47e-07 *** smokerYes 4.9993 0.6975 935.9141 7.168 1.55e-12 *** clinic_urbanurban 1.8470 1.8851 27.6963 0.980 0.336 --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Correlation of Fixed Effects: (Intr) age female smkrYs age -0.555 female -0.162 -0.006 smokerYes -0.077 0.009 0.008 clnc_rbnrbn -0.663 0.002 0.000 0.006 2.5 % 97.5 % (Intercept) 86.3087770 93.7140882 age 0.3682456 0.4470293 female -3.8760676 -1.7338623 smokerYes 3.6322803 6.3663724 clinic_urbanurban -1.8476208 5.5416933 (Intercept) 1 11.199400 2 -1.448419 3 8.886432

Reading the output. The first lines (the REML criterion and the scaled residuals) can be ignored for now. The Random effects block gives the clinic variance (21.57, standard deviation 4.644 mmHg) and the residual variance among patients within a clinic (68.94, standard deviation 8.303 mmHg), and the line below it confirms 966 patients in 30 clinics. The Fixed effects block is read like a regression table: age 0.41 mmHg per year, women 2.80 mmHg lower than men, smokers 5.00 mmHg higher than non-smokers, and urban clinics 1.85 mmHg higher than rural clinics (p = 0.336), each comparing patients with the same values of the other predictors. The df column shows about 937 degrees of freedom for the patient-level predictors and about 28 for urban clinics, because the information about a clinic-level predictor comes from the 30 clinics. The Correlation of Fixed Effects block can be ignored for now.

The 95% confidence intervals agree with the p-values: only the interval for urban clinics (−1.85 to 5.54) includes 0. The intervals from method = "Wald" use the normal distribution, while the lmerTest p-values use a t distribution with the degrees of freedom shown, so for urban clinics the two can differ slightly. The last lines are the estimated shifts of clinics 1 to 3 from the overall average: +11.2, −1.4 and +8.9 mmHg.

vc <- as.data.frame(VarCorr(lmm))
vc[, c("grp", "vcov")]       # variance between and within clinics
vc$vcov[1] / sum(vc$vcov)    # ICC after adjusting for the predictors
Console output
grp vcov 1 clinic_id 21.56638 2 Residual 68.94122 [1] 0.2382825

The two variances are 21.57 between clinics and 68.94 among patients within a clinic, and the last line prints the ICC after adjusting for the predictors, 0.24. The model with no predictors (Activity 5.1) gave a clinic variance of 22.10, a residual variance of 104.34 and an ICC of 0.175.

par(mfrow = c(1, 3))         # three plots side by side
plot(fitted(lmm), resid(lmm), main = "Residuals vs fitted")
abline(h = 0, lty = 2)
qqnorm(resid(lmm), main = "Q-Q: residuals"); qqline(resid(lmm))
re <- ranef(lmm)$clinic_id[, 1]   # the 30 clinic shifts
qqnorm(re, main = "Q-Q: clinic effects"); qqline(re)
par(mfrow = c(1, 1))
isSingular(lmm)              # FALSE: the clinic variance was estimated
Console output
[1] FALSE
Three checks of the linear mixed model: residuals against fitted values form a flat, even cloud; the residuals follow the Q-Q line; the 30 clinic effects follow the Q-Q line except for three high clinics above it.

The three plots check linearity and equal variance (residuals against fitted values), normal residuals (Q-Q plot of the residuals) and normal clinic effects (Q-Q plot of the 30 clinic shifts, where three clinics lie above the line). isSingular() returns FALSE, so the clinic variance was estimated without problems.

lmm_rs <- lmer(sbp ~ age + female + smoker + clinic_urban + (age | clinic_id),
               data = clinics)
Console output
boundary (singular) fit: see help('isSingular')

A random slope, (age | clinic_id), gives each clinic its own age slope as well as its own baseline. R prints a message about a boundary (singular) fit; question 3 asks what it means.

Files for this activity Data Code Answer key

R Reflect on what you just ran

Answer each question from your own output, as on the final exam: find the number, say what it means, judge the check.

1. Interpret the fixed effect for clinic_urbanurban with its 95% confidence interval, saying which patients it compares. Then explain what the estimated shift of 11.2 for clinic 1 means for the patients of that clinic.

Model answerPatients of urban clinics have an average systolic blood pressure 1.85 mmHg higher than patients of the same age, gender and smoking status in rural clinics. The 95% CI (−1.85 to 5.54) includes 0 (p = 0.336), so the data cannot tell urban and rural clinics apart; the comparison is also observational. The interval is wide because urban clinic is a clinic-level predictor: its information comes from the 30 clinics, not the 966 patients (df about 28). Clinic 1's estimated shift of +11.2 mmHg means that its patients average about 11 mmHg higher than patients with the same age, gender, smoking status and clinic type in an average clinic. The shift describes a difference that the predictors in the model do not explain, such as local practices or the population the clinic serves; because everyone in clinic 1 shares it, it does not change the differences between patients within the clinic.

2. Interpret the adjusted ICC that line 24 prints (vc$vcov[1] / sum(vc$vcov), 0.24) in plain words. The model with no predictors (Activity 5.1) had a clinic variance of 22.10, a residual variance of 104.34 and an ICC of 0.175. Use the two variances of each model, which line 23 prints, to explain why the adjusted ICC is larger.

Model answerLine 24 prints an adjusted ICC of 0.24 (0.2382825): the clinic variance as a share of the total of the two variances. After allowing for age, gender, smoking and clinic type, about 24% of the remaining variation in systolic blood pressure lies between clinics, so patients in the same clinic are still clearly alike. Age, gender and smoking explain much of the variation among patients within clinics, so the residual variance falls from 104.34 to 68.94. They explain very little of the variation between clinics, which stays at about 22 (22.10 against 21.57), so the clinic share of the remaining variation rises from 0.175 to 0.24; a strong clinic-level predictor could instead shrink the clinic variance and lower the ICC.

3. Run your own code: at the end of the script, type VarCorr(lmm_rs) and fixef(lmm_rs) and run them. First, name two checks you ran on lmm and say what your output shows for them. Then explain what the random slope for age in lmm_rs would let the model describe, and what the message from line 37, the VarCorr() output and the fixed effects suggest you should do.

Model answerThe plot of residuals against fitted values is a flat, even cloud and the residuals follow the Q-Q line, which support linearity, equal variance and normal residuals. The Q-Q plot of the clinic effects follows the line except for three clinics (1, 3 and 11) with higher baselines than a normal distribution predicts, which are worth reporting and investigating rather than a reason to abandon the model. There are 30 clinics, which meets the usual minimum of about 20 to 30 clusters, and isSingular(lmm) is FALSE, so the clinic variance was estimated and the random intercept is supported. The random slope (age | clinic_id) would let the association between age and blood pressure differ from clinic to clinic, which suits a question about how an effect varies between settings. R reports a boundary (singular) fit, and VarCorr(lmm_rs) shows a standard deviation of only 0.014 for the age slopes and a correlation of −1.000, so the random part of the model is more complex than 30 clinics can support. Report the random-intercept model and state that a random slope for age could not be estimated; the fixed effects barely change (smoking 5.00 in both models), so the conclusion does not depend on the random slope.
Saved.

Your output from Activity 5.2 shows the fixed effects, the adjusted ICC and the model checks interpreted in this section, together with the singular fit that a random slope produces with 30 clinics. The key takeaways below draw the section together before its knowledge check.

Key Takeaways

  • 4.1 A random-intercept model is a regression in which each cluster gets its own baseline. In lmer() you add the term (1 | clinic_id), where the 1 stands for the intercept and the bar can be read as "for each", so the term means "an intercept for each clinic". Each clinic's intercept is the overall intercept plus a shift for that clinic, and rather than estimating 30 separate shifts, the model treats them as a random sample from a normal distribution and summarises them with one variance. This is how it differs from an ordinary regression, which has a single intercept and treats every patient as independent: the mixed model allows for the clustering and can still estimate a clinic-level predictor such as clinic_urban, with a standard error that reflects the 30 clinics. If a question asks what (1 | cluster) adds, say that it gives each cluster its own baseline and allows for the fact that people in the same cluster are alike.
  • 4.2 The fixed effects of a mixed model are read just like the coefficients of an ordinary linear regression: each is a difference in the average outcome between people with the same values of the other predictors in the same cluster. For example, the coefficient of 5.00 for smokerYes means that smokers have an average systolic blood pressure 5.0 mmHg higher than non-smokers of the same age and gender attending the same clinic (95% CI 3.63 to 6.37). An estimated cluster shift is read for the whole cluster: patients of a clinic with a shift of +8 mmHg average about 8 mmHg higher than similar patients in an average clinic, a difference that the predictors in the model do not explain. Because everyone in a clinic shares its shift, it raises every prediction for that clinic by the same amount but leaves the difference between two of its patients unchanged, as Worked Example 5.2 shows. In your own interpretations, say which people are being compared, give the units and the confidence interval, and if the interval for a cluster-level predictor includes 0, say that the data cannot tell the groups apart.
  • 4.3 The random-effects block of summary() gives two variances: the variance between clusters (the clinic_id (Intercept) row, 21.57) and the residual variance among people within a cluster (the Residual row, 68.94). Their square roots, the standard deviations (4.64 and 8.30 mmHg), are in the units of the outcome, which makes them easier to picture. The adjusted ICC comes from the same line of R code as before, vc$vcov[1] / sum(vc$vcov), which prints 0.24 (the clinic share of 21.57 + 68.94), and describes the clustering that remains once the predictors are in the model. It can be larger or smaller than the ICC from the model with no predictors: here age, gender and smoking explained much of the variation among patients within clinics (the residual variance fell from about 104 to 69) but little of the variation between clinics (about 22 in both models), so the ICC rose from 0.175 to 0.24, whereas a strong cluster-level predictor can shrink the cluster variance and lower the ICC. To explain any change in the ICC, look at which of the two variances fell and by how much.
  • 5.1 Before you trust a mixed model, run a short set of checks, and in an exam be ready to name them and say what a problem would look like. A plot of residuals against fitted values should be a flat, even cloud (linearity and equal variance), and a Q-Q plot of the residuals should follow the line (normal residuals). A Q-Q plot of the cluster effects from ranef() checks that the cluster shifts are roughly normal; it is rough with few clusters, and a few clusters off the line, like the three high clinics in Figure 5.5, are worth reporting and investigating rather than a reason to abandon the model. You also need enough clusters, about 20 to 30 or more (the groups line of the output or nlevels() tells you how many), clusters that are independent by design, and isSingular() returning FALSE, which means the cluster variance was estimated rather than collapsing to zero. When an output reports one of these checks, say what it shows and whether the model is acceptable, rather than only naming the check.
  • 5.2 A random slope lets the effect of a predictor, and not only the baseline, differ from cluster to cluster. In lmer(), (age | clinic_id) gives each clinic its own intercept and its own age slope, which suits research questions about how an effect varies between settings, such as whether a program works better in some schools than in others. Random slopes need more clusters and more observations per cluster, so with about 30 clusters they often produce a warning. A convergence warning means that R stopped before it was sure it had found the best estimates, and a singular fit means that a variance was estimated at or near zero (or a correlation at exactly ±1); both say that the random part of the model is more complex than the data can support. The sensible response, and the one exam answers look for, is to report the simpler random-intercept model, state that a random slope could not be estimated, and note that if the main fixed effect is almost the same in both models, the conclusion does not depend on the random slope.
Knowledge check: this section

1. A study of 1,020 residents in 34 long-term care homes fits lmer(qol ~ age10 + dementia + therapist + (1 | home_id)), where qol is a quality-of-life score and therapist records whether the home employs a recreation therapist. The fixed effect for dementia is −3.1 points, and the estimated shift (random effect) for home H07 is +4.0 points. Which statement is correct?

A random intercept does not add 34 fixed coefficients. Each home's shift is treated as a draw from a distribution, and the model summarises the shifts with one home variance.
This confuses a random intercept with a random slope. (1 | home_id) shifts the baseline of every resident in a home equally, so the dementia difference stays the same; a varying dementia difference would need (dementia | home_id).
The shift describes how H07 differs from an average home in ways the predictors do not explain. The therapist is a measured predictor with its own fixed coefficient, so the shift cannot be attributed to it.
A random intercept gives each home its own baseline, summarised by one variance. H07's shift raises every prediction in that home equally, so it cancels out of comparisons within H07 and the dementia difference is unchanged.
Correct answer: the term is a random intercept, and H07's shift raises the baseline of everyone in that home by about 4 points. The information needed is the meaning of (1 | home_id), read as “an intercept for each home”, and H07's estimated shift. The shift moves every prediction for H07 up by 4 points, so it cancels out of any comparison between two residents of H07 and the dementia difference stays −3.1; it describes a difference between homes that the predictors do not explain, not the effect of a measured variable such as the therapist, which has its own coefficient. The option that adds 4.0 to the dementia effect confuses a random intercept with a random slope, which would be written (dementia | home_id). (Section 2, 4.1 The Random-Intercept Model and Equation 5.3, and 4.2 Interpreting the Fixed Effects and Cluster Shifts with Worked Example 5.2; Objectives 4.1 and 4.2)

2. The fixed effects of the care-home model are shown below. Seventeen of the 34 homes employ a recreation therapist, and the homes chose this themselves. Which is the best interpretation of the therapist coefficient?

Fixed effectEstimate95% CIp-value
Age, per 10 years−0.9−1.3 to −0.5< 0.001
Dementia−3.1−4.0 to −2.2< 0.001
Recreation therapist2.4−0.5 to 5.30.12
A fixed effect is read like an ordinary coefficient, as an adjusted difference in average outcome, and the interval including 0 means the data are compatible with no difference; because homes chose to employ a therapist, it is an association.
This causal reading ignores two things: the 95% CI includes 0, so the data are compatible with no difference, and the homes chose whether to employ a therapist, so the comparison is observational.
A positive estimate that is larger than another coefficient is not evidence of a clear difference. The interval for the therapist includes 0, so the 34 homes cannot show that the difference is real.
A fixed coefficient is a difference in average outcome in points, not a percentage of variation explained. Shares of variation come from the variance components, not from a coefficient.
Correct answer: about 2.4 points higher, with a confidence interval that includes 0. The information needed is the estimate (2.4), its 95% confidence interval (−0.5 to 5.3) and the fact that the therapist is a home-level characteristic that the homes chose for themselves. A fixed effect is read like an ordinary regression coefficient, as a difference in the average outcome for residents with the same values of the other predictors; because the interval includes 0 (p = 0.12), the data are compatible with no difference, and a difference between observed homes is an association, not a proven effect. The causal reading is tempting, but it ignores both the interval and the observational design. (Section 2, 4.2 Interpreting the Fixed Effects and Cluster Shifts and Table 5.4; Objective 4.2)

3. In the care-home study, the model with no predictors gave a home variance of 9.0, a residual variance of 66.0 and an ICC of 0.12. For the model with age, dementia and therapist, the analyst obtained the variances and the adjusted ICC in R as shown below. What does the adjusted ICC mean, and why does it differ from 0.12?

> vc <- as.data.frame(VarCorr(m_qol)) > vc[, c("grp", "vcov")] grp vcov 1 home_id 8.031562 2 Residual 31.968120 > vc$vcov[1] / sum(vc$vcov) # adjusted ICC [1] 0.2007906
The value is read correctly but the reason is wrong. Predictors do not create clustering; the adjusted ICC rose because the predictors explained much of the within-home variation and little of the between-home variation.
This misreads the rows. The residual variance is the variation among residents within homes, so the between-home share is the home variance as a share of the total, which is about 20%, not 80%.
The adjusted ICC is the home share of the variation left after the predictors. It rose because the predictors reduced the within-home variance much more than the home variance, so homes account for a larger share of what remains.
The random intercept stays in the model whatever predictors are added. The rise in the ICC reflects which variance the predictors explained, not a model that stopped allowing for clustering.
Correct answer: about 20% of the remaining variation lies between homes, and the ICC rose because the predictors mainly reduced the within-home variance. The information needed is the adjusted ICC that R printed (0.20, the home share of the two variances) and the variances of the two models. Comparing them shows which variance the predictors explained: the residual variance fell by about half (66.0 to 32.0) while the home variance barely changed (9.0 to 8.0), so the home share of the remaining variation rose from 0.12 to 0.20; had the predictors explained a larger share of the home variance than of the residual variance, the adjusted ICC would have been smaller. The residual row is the variation within homes, not between them, and predictors do not create clustering or switch off the random intercept. (Section 2, 4.3 The Variance Components and the Adjusted ICC; Objective 4.3)

4. The analyst checks the care-home model and reports: the plot of residuals against fitted values is a flat cloud with an even spread; the Q-Q plot of the residuals follows the line; the Q-Q plot of the 34 home effects follows the line except for one home well above it; nlevels(home_id) is 34; and isSingular(model) returns FALSE. What should the analyst conclude?

One cluster above the line is a mild departure from normal cluster effects, and fixed effects are not very sensitive to it. The appropriate response is to report and investigate that home, not to discard a model that passes its other checks.
Each check gave the result it should: residual plots support linearity, equal variance and normality, 34 homes meet the usual 20 to 30 minimum, and isSingular() returning FALSE shows the home variance was not estimated as zero.
This reverses the meaning of the check. isSingular() returns TRUE when a variance is estimated at or near zero; FALSE is the desired result, showing that the home variance was estimated.
The lesson gives about 20 to 30 clusters as a common minimum for a mixed model, not 100. With 34 homes the home variance can be estimated adequately.
Correct answer: the model is acceptable, with the one high home noted. The information needed is the result of each check and what it should show: a flat, even cloud supports linearity and equal variance, points on the line support normal residuals and normal home effects, about 20 to 30 clusters is a common minimum, and isSingular() should return FALSE, which means the home variance was not estimated as zero. One home above the line is a mild departure from normal cluster effects, and fixed effects are not very sensitive to it, so it is reported and the home examined rather than the model discarded. The option about isSingular() reverses the meaning of the check. (Section 2, 5.1 Checking a Linear Mixed Model, Table 5.5 and Figure 5.5; Objective 5.1)

5. To ask whether the dementia difference varies between homes, the analyst fits lmer(qol ~ age10 + dementia + therapist + (dementia | home_id)). R prints “boundary (singular) fit”, and the dementia estimate is almost unchanged (−3.0 against −3.1). Which statement is correct?

Agreement in the fixed effects does not make a singular fit acceptable. The warning means the random part is more complex than the data support, so the simpler random-intercept model is reported.
A variance estimated at or near zero shows that 34 homes cannot measure variation in the dementia difference, not that there is none. A singular fit is a limitation of the data, not a finding.
A singular fit concerns the random part of the model, a variance at or near zero or a correlation at its limit; it says nothing about the normality of residuals and is not a reason to switch to glmer().
A random slope lets an effect differ between clusters, but a singular fit shows the data cannot support that extra variance. Since the dementia estimate barely changes, the random-intercept model is reported with a note.
Correct answer: report the random-intercept model and note that the random slope could not be estimated. The information needed is what (dementia | home_id) adds (each home gets its own dementia slope as well as its own baseline, which suits a question about how an effect varies between settings) and what the message means. A singular fit says that the random part of the model is more complex than the data can support, so a variance sits at or near zero (or a correlation at ±1); because the dementia estimate barely changes, the conclusion does not depend on the random slope. The tempting reading that the dementia difference is identical in every home is wrong: a variance estimated at zero shows that 34 homes cannot measure the variation, not that there is none. (Section 2, 5.2 Random Slopes and Model Warnings; Objective 5.2)

6. A researcher studies body mass index (BMI) among 1,200 employees of 15 workplaces. Seven of the workplaces have an on-site cafeteria that serves subsidised healthy meals (cafeteria = 1), a characteristic of the workplace. The model m1 <- lmer(bmi ~ age10 + female + cafeteria + (1 | workplace), data = staff) gives the output below. What do these lines show?

> nlevels(staff$workplace) [1] 15 > isSingular(m1) [1] FALSE > round(summary(m1)$coefficients["cafeteria", ], 3) Estimate Std. Error df t value Pr(>|t|) -1.300 0.700 12.100 -1.857 0.088
Passing the singular-fit check shows the variance was estimated, not that it was estimated precisely. The cluster variance and a cluster-level predictor draw their information from the clusters, and 15 workplaces is below the usual minimum.
The number of employees does not make up for few clusters. The workplace variance and the cafeteria effect are estimated from the workplaces, so 15 workplaces limit their precision whatever the number of employees.
The df column reflects the number of independent units informing a workplace-level predictor, which is the workplaces, not a count of employees. Every employee contributes, but the information comes from 15 workplaces.
isSingular() only shows that the workplace variance was not estimated as zero. Whether there are enough clusters is a separate check, counted with nlevels(), and 15 is below the usual 20 to 30.
Correct answer: the fit is not singular, but 15 workplaces are too few for precise estimates. The information needed is what each check shows: isSingular() returning FALSE means that the workplace variance was not estimated as zero, while nlevels() counts the clusters, and about 20 to 30 is a common minimum because the cluster variance is estimated from the clusters themselves. The cafeteria is a workplace-level predictor, so its information comes from the 15 workplaces, which is why the df column shows about 12 rather than about 1,200, just as urban clinic had about 28 for 30 clinics. The number of employees does not make up for few clusters, and passing the singular-fit check does not show that there are enough of them. (Section 2, 4.2 Interpreting the Fixed Effects and Cluster Shifts (the df column), 5.1 Checking a Linear Mixed Model, Table 5.5 and the slide “Enough clusters, and no singular fit”; Objective 5.1)
● Complete the knowledge check to continue.
Section 3 of 4

Binary Outcomes: Mixed Models and GEE

⏱ Estimated time: 35 minutes
Lesson 5 · Section 3

Binary Outcomes: Mixed Models and GEE

Two ways to allow for clustering when the outcome is yes or no.

The outcome

Specialist referral in 30 clinics

246
of 966 patients referred
25%
overall referral rate
30
clinics, with different referral rates

Referral is binary, so each method is a version of logistic regression that allows for the clinics.

Approach 1

The GLMM: logistic regression with a random intercept

glmer(referred ~ age + smoker + clinic_urban + (1 | clinic_id), family = binomial, data = clinics)

What it estimates

The odds ratios compare two patients in the same clinic, or in two clinics with the same baseline, so they are clinic-specific (conditional) odds ratios.

Clinic variation

The clinic shifts on the log-odds scale have a variance of 0.30 (standard deviation 0.55).

Approach 2

GEE: a population-averaged model with corrected standard errors

geeglm(referred ~ age + smoker + clinic_urban, id = clinic_id, family = binomial, corstr = "exchangeable", data = clinics)

What it estimates

The odds ratios describe the population as a whole, so they are population-averaged (marginal) odds ratios.

Working correlation

An exchangeable structure assumes that any two patients in a clinic are equally correlated, and here the correlation is estimated at 0.04.

Comparing the models

Smoking barely changes; urban clinic does

Odds ratios with 95% confidence intervals from three models. Smoking: 1.72, 1.67 and 1.64 with similar intervals. Urban clinic: 2.16 from glm, 2.31 from glmer and 2.04 from geeglm, with wider intervals for the last two.
The standard errors for urban clinic on the log scale are 0.17 for glm, 0.28 for the GLMM and 0.29 for GEE.
Two questions

Clinic-specific or population-averaged?

GLMM (clinic-specific)

This compares patients in the same clinic, or in clinics with the same baseline. Its odds ratios are usually a little further from 1. It suits questions about individual clinics.

GEE (population-averaged)

This compares referral across the whole population, averaged over all clinics. It suits questions about policy for the population.

For a continuous outcome, as in Section 2, the two kinds of estimate coincide.

Assumptions and checks

What to check in a GLMM and in GEE

CheckGLMM: glmer()GEE: geeglm()
Binary outcome and enough eventsYes (about 10 per predictor)Yes
Enough clustersAbout 20 to 30About 30 to 40 or more
Cluster effectsAssumed normal (log-odds scale)No assumption about their shape
WarningsRead convergence warningsCheck the cluster id and sort the data
Choosing a method

GLMM or GEE?

QuestionPoints to
Is the question about individual clusters?GLMM
Is the question about the whole population?GEE
Are there fewer than about 30 clusters?GLMM
Is the variation between clusters of interest?GLMM
Is the outcome continuous?Linear mixed model (Section 2)

Both methods agree here: urban clinics refer more patients.

Carry forward

What to take into the next section

  • A GLMM adds a random intercept to a logistic regression, and GEE fits a population-averaged logistic regression with standard errors that allow for clustering.
  • The GLMM gives clinic-specific odds ratios and GEE gives population-averaged odds ratios.
  • Both methods widen the standard errors of cluster-level predictors, and GEE needs more clusters.

Overview

This section extends logistic regression to clustered data. The outcome is referred, which records whether each of the 966 patients in phaa_clinics.csv was referred to a specialist (246 were, as table(clinics$referred) in Activity 5.3 shows). The predictors are age, smoking and urban clinic. A generalized linear mixed model (GLMM) and a generalized estimating equations (GEE) model are fitted and compared with an ordinary logistic regression. The two approaches allow for clustering in different ways and estimate slightly different odds ratios, so the section also explains what each kind of odds ratio means, how to choose between the approaches for a given research question, and how each is checked.

Learning Objectives

  • 6.1 Describe how a logistic GLMM (glmer()) and a logistic GEE model (geeglm()) each allow for clustering.
  • 6.2 Interpret odds ratios from both models and compare their standard errors with an ordinary logistic regression.
  • 6.3 Distinguish clinic-specific (conditional) from population-averaged (marginal) odds ratios in plain language.
  • 6.4 Choose between a GLMM and GEE using the research question and the number of clusters.
  • 6.5 List the checks for each approach.

6.1 How a GLMM and GEE Allow for Clustering

Logistic regression can allow for clustering in two ways, which correspond to the two families of methods introduced in Section 1 and listed in Table 5.3. A GLMM gives each clinic its own baseline log-odds, and GEE fits one model for the whole population and calculates standard errors that allow for the clustering.

A generalized linear mixed model (GLMM) adds random effects to a generalized linear model. For a binary outcome, it is a logistic regression in which each clinic has its own baseline log-odds, with the clinic shifts drawn from a normal distribution, just as in the linear mixed model of Section 2. It is fitted with glmer() from lme4, which takes the same family = binomial argument as glm(), and the random intercept (1 | clinic_id) goes inside the model formula. Equation 5.4 writes the GLMM for referral in the clinic data.

Logistic GLMM for referral
\[ \ln\frac{\color{#0B7B6B}{p_{ij}}}{1 - \color{#0B7B6B}{p_{ij}}} = \color{#6D28D9}{\beta_0} + \color{#C2410C}{u_j} + \color{#1D4ED8}{\beta_1}\,\text{age}_{ij} + \color{#1D4ED8}{\beta_2}\,\text{smoker}_{ij} + \color{#1D4ED8}{\beta_3}\,\text{urban}_{j} \]Eq 5.4
The log-odds of referral for patient i in clinic j equals the overall intercept plus a clinic shift plus the fixed effects. Exponentiating a fixed effect gives an odds ratio for patients in the same clinic, or in clinics with the same baseline.

Generalized estimating equations (GEE) take a different approach. They fit a logistic regression without clinic effects, so no clinic is given its own baseline. Instead, GEE deals with the clustering in two steps. First, a working correlation describes how alike patients in the same clinic are assumed to be, and GEE uses it to weight the data. By this we mean that if patients in the same clinic are assumed to be correlated, each additional patient from a clinic is treated as adding a little less new information, so the patients of a large clinic count for slightly less each. Second, the standard errors are calculated from how much the data actually vary from clinic to clinic, using the totals of the residuals within each clinic, rather than from the variance the model assumes. Because these standard errors are built from the clinics, they allow for the clustering (Zeger & Liang, 1986). The analyst chooses the working correlation structure. For people in clinics, schools or households, the usual choice is "exchangeable", which assumes that any two members of a cluster are equally correlated. These corrected standard errors are called sandwich (or robust) standard errors, after the shape of the formula used to calculate them. Their key strength is that they remain valid even if the working correlation is not exactly right, provided there are enough clusters, because they rely on the variation observed between clusters rather than on the assumed correlation. GEE is fitted with geeglm() from geepack, which needs the cluster identifier in the id argument rather than in the formula, and the data sorted so that each cluster's rows are together.

In short, the GLMM allows for clustering with a random intercept and GEE with sandwich standard errors. The next part fits both models to the referral data and compares their results with an ordinary logistic regression.

6.2 Odds Ratios and Standard Errors from the Three Models

Activity 5.3 at the end of this section fits an ordinary logistic regression (glm()), a GLMM (glmer()) and GEE (geeglm()) with the same three predictors. Odds ratios from both clustered models are obtained and read as in ordinary logistic regression, by exponentiating the coefficients with exp(fixef(glmm)) or exp(coef(gee)). Table 5.6 shows the odds ratios for smoking (a patient-level predictor) and urban clinic (a clinic-level predictor), with their standard errors on the log-odds scale.

Table 5.6. Odds ratios for smoking and urban clinic, with standard errors on the log-odds scale, from three models of specialist referral.

ModelOR for smokingSE (smoking)OR for urban clinicSE (urban clinic)
Ordinary logistic regression, glm()1.720.1882.160.168
GLMM, glmer()1.670.1942.310.279
GEE, geeglm()1.640.1862.040.289

Figure 5.6 plots the same odds ratios with their 95% confidence intervals.

Odds ratios with 95% confidence intervals from glm, glmer and geeglm. Smoking: 1.72, 1.67 and 1.64 with similar intervals. Urban clinic: 2.16, 2.31 and 2.04, with the glmer and geeglm intervals much wider than the glm interval.
Figure 5.6. The left panel shows the odds ratios for smoking and the right panel the odds ratios for urban clinics, each with a 95% confidence interval. Allowing for clinics barely changes the smoking result but widens the interval for urban clinics.

For smoking, the three models give similar odds ratios and standard errors, because smokers and non-smokers are found in every clinic and the comparison can be made within clinics. For urban clinic, the standard error rises from 0.168 in the ordinary logistic regression to 0.279 in the GLMM and 0.289 in GEE. This is the same pattern as for blood pressure in Section 1, with the same explanation: urban clinic has only 30 independent values, whereas smoking can be compared within every clinic. The odds ratio remains clearly above 1 in all three models: patients in urban clinics have about twice the odds of referral of patients in rural clinics of the same age and smoking status (GLMM OR 2.31, p = 0.003; GEE OR 2.04, p = 0.014).

The GEE odds ratio for urban clinics (2.04) is a little lower than the glm() odds ratio (2.16) because the exchangeable working correlation gives each patient in a large clinic slightly less weight. This happens because, when patients in a clinic are assumed to be correlated, the 45th patient from a clinic adds less new information than the first, so large clinics influence the estimate a little less than they do in glm(), which counts every patient equally. As a result, the balance of urban and rural clinics in the estimate shifts slightly, and the odds ratio moves with it. Refitting GEE with corstr = "independence" reproduces the glm() odds ratio of 2.16, with the corrected standard error of 0.289.

Both clustered models therefore widen the standard error of the clinic-level predictor and agree that urban clinics have about twice the odds of referral. Their odds ratios still differ (2.31 against 2.04), because they answer slightly different questions, which the next part explains.

6.3 Clinic-Specific and Population-Averaged Odds Ratios

A GLMM odds ratio is clinic-specific, or conditional: it compares two patients in the same clinic, or in clinics with the same baseline, so the GLMM's 2.31 compares similar patients in an urban and a rural clinic with the same baseline. By this we mean that the clinic's own baseline is held fixed, and the odds ratio compares the odds of referral in an urban and a rural clinic for patients whose clinic baselines are the same. A GEE odds ratio is population-averaged, or marginal: it compares all urban-clinic patients with all rural-clinic patients of the same age and smoking status, across every clinic. In this case the clinic baselines are not held fixed; instead, the proportion referred is averaged over all the clinics, with their high and low baselines mixed together, before the two groups are compared. The two kinds of odds ratio therefore answer different questions. The first is about the comparison within a clinic, and the second is about the difference that would be seen in the population as a whole. Because odds ratios do not average in a simple way, these two questions generally have different answers whenever the clinics differ in their baselines, as Worked Example 5.3 shows. An interpretation of either odds ratio should name which kind it is, for example by beginning "for two patients in the same clinic" for a GLMM result and "across the whole population, averaging over clinics" for a GEE result. Click each of the cards below to compare the two kinds of odds ratio and the questions they answer, and to review the working correlation that GEE uses.

Clinic-Specific Odds RatioClick to explore
Population-Averaged Odds RatioClick to explore
Working CorrelationClick to explore

Worked Example 5.3 explains why the GLMM odds ratio for urban clinics in Table 5.6 is further from 1 than the GEE odds ratio. Watch how averaging over clinics with different baselines changes the comparison.

Worked Example 5.3: Why the GLMM Odds Ratio Is Further From 1

Suppose that a patient in an urban clinic has 2.3 times the odds of referral of a similar patient in a rural clinic with the same baseline. Clinics also differ in their baseline odds, so the population of urban patients mixes clinics with high and low baselines, and so does the population of rural patients. Averaging the probability of referral over this mix pulls the population comparison toward 1, so the population-averaged odds ratio is smaller than the clinic-specific odds ratio. The reason is that the same odds ratio produces different gaps in probability at different baselines. In a clinic where almost no one is referred, or almost everyone is, multiplying the odds by 2.3 moves the probability of referral only a little, whereas in a clinic with a middling baseline it moves the probability a lot. When the probabilities from all of these clinics are averaged together, the small gaps dilute the large ones, and the resulting population comparison, expressed as an odds ratio, is closer to 1 than the clinic-specific odds ratio. Averaging over clinics is one reason the GEE odds ratio (2.04 here) is smaller than the GLMM odds ratio (2.31), and the weighting used by GEE and chance variation also contribute in this sample. The larger the variation between clinics, the larger the gap. For a continuous outcome with an identity link, averaging does not change the difference, so the two kinds of estimate are the same.

Neither model is wrong when the two odds ratios differ: each answers a valid question, and the gap reflects how much the clinics vary. Which question matters depends on who is asking, which is the basis for choosing between the two approaches.

6.4 Choosing Between a GLMM and GEE

The choice depends mainly on the research question (Subramanian & O'Malley, 2010). A GLMM suits questions about individual clusters, estimates how much the clusters vary, and works with fewer clusters. GEE suits questions about the population as a whole and makes no assumption about the distribution of cluster effects (Hubbard et al., 2010), but it needs more clusters for reliable standard errors (Hemming & Taljaard, 2023). Published rules of thumb for the number of clusters vary. This lesson uses about 20 to 30 clusters for a mixed model and about 30 to 40 or more for GEE as working values, while Hemming and Taljaard advise a small-sample correction whenever a GLMM has fewer than about 40 clusters or a GEE analysis fewer than about 50. The need for more clusters follows from how GEE works: because its sandwich standard errors are calculated from the variation observed between clusters, each cluster acts as one piece of evidence about that variation, and with only a few clusters the standard errors themselves are estimated imprecisely and tend to be too small. For example, a clinic manager asking how much higher the odds of referral are for a patient in her own clinic needs the GLMM's clinic-specific odds ratio, while a provincial planner asking how referral differs across the province needs the GEE population-averaged odds ratio. The number of clusters then has to be checked: a GLMM works with about 20 to 30 clusters, while GEE needs about 30 to 40 or more, so the 30 clinics here meet the GLMM requirement and sit at the lower limit for GEE. In practice, many epidemiological papers report one approach as the main analysis and the other as a sensitivity analysis. When they agree, as they do for urban clinics here, the conclusion is stronger.

Whichever approach is chosen, its result rests on assumptions that need to be checked before it is reported, as the next part sets out.

6.5 Checking a GLMM and GEE

The GLMM and GEE share some assumptions and differ in others, including the number of clusters each needs. Table 5.7 lists the assumptions of each approach with how to check them.

Table 5.7. Assumptions of the logistic GLMM and GEE, with how to check them.

AssumptionGLMMGEEHow to check it
Binary outcome with enough eventsRequiredRequiredtable(outcome); about 10 events per predictor (246 events here).
Independent clustersRequiredRequiredJudged from the study design.
Enough clustersAbout 20 to 30About 30 to 40 or morenlevels(clinic_id); 30 clinics here, at the lower limit for GEE.
Normal cluster effects (log-odds scale)AssumedNot assumedQ-Q plot of ranef(model).
Model fitted without problemsRead any convergence warningsCorrect id, data sorted by clusterWarnings in the console; the "Number of clusters" line in the GEE output.

GEE has no clinic effects, so the Q-Q plot of the cluster effects is a check for the GLMM only. GEE instead requires one step before the model is fitted, and forgetting it is a common and silent error, as the caution below explains.

Sort the Data Before Using geeglm()

geeglm() treats consecutive rows with the same id as one cluster. If the rows of a clinic are scattered through the file, the function splits that clinic into several clusters and the standard errors are wrong without any warning. Sorting the data by the cluster identifier (and by time for repeated measures) before fitting avoids the problem. The line Number of clusters: 30 in the output confirms that the clinics were read correctly.

A GLMM and GEE therefore both allow for clustering in a binary outcome, widen the standard error for the clinic-level predictor and reach the same conclusion for urban clinics, while answering clinic-specific and population-averaged questions respectively. Activity 5.3 lets you fit the three models in Table 5.6 in R, compare them and see what happens when the data are not sorted: run the script and answer the questions from your own output. Section 4 then applies a linear mixed model and GEE to repeated measurements of the same people.

R Activity 5.3: a GLMM and GEE for specialist referral

Text version of the activity: the code of the run-along script, its console output and notes on reading it.

library(lme4); library(geepack)   # glmer() for the GLMM, geeglm() for GEE
clinics <- read.csv("https://heal-learn.ca/r-activities/data/phaa_clinics.csv")   # course data, read from the website
clinics$clinic_id <- factor(clinics$clinic_id)
clinics$smoker <- factor(clinics$smoker, levels = c("No", "Yes"))   # "No" = reference
clinics$clinic_urban <- factor(clinics$clinic_urban, levels = c("rural", "urban"))
table(clinics$referred)        # 1 = referred: how many events?
nlevels(clinics$clinic_id)     # how many clusters?
Console output
Loading required package: Matrix 0 1 720 246 [1] 30

The two checks before fitting: 246 of the 966 patients were referred, far more than the guide of about 10 events per predictor for three predictors, and there are 30 clinics, enough for a GLMM (about 20 to 30 clusters) and at the lower limit for GEE (about 30 to 40 or more).

naive <- glm(referred ~ age + smoker + clinic_urban, family = binomial, data = clinics)

The ordinary logistic regression treats the 966 patients as independent. Its odds ratios and standard errors appear in the comparison at the end of the activity.

glmm <- glmer(referred ~ age + smoker + clinic_urban + (1 | clinic_id),
              family = binomial, data = clinics)
summary(glmm)
round(exp(cbind(OR = fixef(glmm), confint(glmm, parm = "beta_", method = "Wald"))), 3)
Console output
Generalized linear mixed model fit by maximum likelihood (Laplace Approximation) [glmerMod] Family: binomial ( logit ) Formula: referred ~ age + smoker + clinic_urban + (1 | clinic_id) Data: clinics AIC BIC logLik -2*log(L) df.resid 1019.6 1044.0 -504.8 1009.6 961 Scaled residuals: Min 1Q Median 3Q Max -1.7792 -0.6038 -0.4221 0.7081 3.6911 Random effects: Groups Name Variance Std.Dev. clinic_id (Intercept) 0.3023 0.5498 Number of obs: 966, groups: clinic_id, 30 Fixed effects: Estimate Std. Error z value Pr(>|z|) (Intercept) -3.943363 0.425514 -9.267 < 2e-16 *** age 0.040244 0.006203 6.488 8.69e-11 *** smokerYes 0.511383 0.193868 2.638 0.00834 ** clinic_urbanurban 0.835563 0.278527 3.000 0.00270 ** --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Correlation of Fixed Effects: (Intr) age smkrYs age -0.833 smokerYes -0.135 0.036 clnc_rbnrbn -0.498 0.056 0.025 OR 2.5 % 97.5 % (Intercept) 0.019 0.008 0.045 age 1.041 1.028 1.054 smokerYes 1.668 1.140 2.438 clinic_urbanurban 2.306 1.336 3.981

Reading the output. In the GLMM, the Random effects block gives a clinic variance of 0.30 on the log-odds scale (standard deviation 0.55), so clinics differ in their baseline odds of referral. The Fixed effects block is read like a logistic regression: each coefficient is a log odds ratio for patients in the same clinic or, for urban clinic, in clinics with the same baseline. The urban-clinic coefficient is 0.836 (SE 0.279, p = 0.003), and the last line prints its odds ratio, 2.31 (95% CI 1.34 to 3.98): a clinic-specific (conditional) odds ratio.

clinics <- clinics[order(clinics$clinic_id), ]   # each clinic's rows together
gee <- geeglm(referred ~ age + smoker + clinic_urban, id = clinic_id,
              family = binomial, corstr = "exchangeable", data = clinics)
summary(gee)
round(exp(cbind(OR = coef(gee), confint.default(gee))), 3)
Console output
Call: geeglm(formula = referred ~ age + smoker + clinic_urban, family = binomial, data = clinics, id = clinic_id, corstr = "exchangeable") Coefficients: Estimate Std.err Wald Pr(>|W|) (Intercept) -3.669981 0.369228 98.796 < 2e-16 *** age 0.038053 0.004334 77.081 < 2e-16 *** smokerYes 0.492949 0.185834 7.036 0.00799 ** clinic_urbanurban 0.711418 0.288681 6.073 0.01373 * --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Correlation structure = exchangeable Estimated Scale Parameters: Estimate Std.err (Intercept) 0.9896 0.1111 Link = identity Estimated Correlation Parameters: Estimate Std.err alpha 0.04219 0.0157 Number of clusters: 30 Maximum cluster size: 45 OR 2.5 % 97.5 % (Intercept) 0.025 0.012 0.053 age 1.039 1.030 1.048 smokerYes 1.637 1.137 2.356 clinic_urbanurban 2.037 1.157 3.587

The GEE output gives population-averaged coefficients with corrected (sandwich) standard errors in the Std.err column and Wald tests in the next two columns. The urban-clinic coefficient is 0.711 (SE 0.289, p = 0.014), an odds ratio of 2.04 (95% CI 1.16 to 3.59): a population-averaged (marginal) odds ratio. The estimated exchangeable correlation (alpha) is 0.04, and the line Number of clusters: 30 confirms that the sorted data were read as 30 clinics. The Estimated Scale Parameters block can be ignored. Printing summary() of a GEE model also lowers the number of digits R prints for the rest of the session, which is why later output is shorter.

or_se <- function(fit, x) {
  b <- summary(fit)$coefficients[x, 1:2]   # estimate and its SE
  c(OR = exp(b[[1]]), SE = b[[2]])
}
three <- function(x) rbind(glm = or_se(naive, x), glmer = or_se(glmm, x), geeglm = or_se(gee, x))
round(three("smokerYes"), 3)           # a patient-level predictor
round(three("clinic_urbanurban"), 3)   # a clinic-level predictor
Console output
OR SE glm 1.718 0.188 glmer 1.668 0.194 geeglm 1.637 0.186 OR SE glm 2.165 0.168 glmer 2.306 0.279 geeglm 2.037 0.289

For smoking, a patient-level predictor, the three models give similar odds ratios and standard errors (0.188, 0.194 and 0.186). For urban clinic, a clinic-level predictor, the odds ratio is about 2 in all three models, but its standard error rises from 0.168 when the clinics are ignored to 0.279 (GLMM) and 0.289 (GEE).

Files for this activity Data Code Answer key

R Reflect on what you just ran

Answer each question from your own output, as on the final exam: interpret each odds ratio and say what kind it is, explain the standard errors, and judge the checks before you choose a result to report.

1. Interpret the odds ratio for clinic_urbanurban from the GLMM and from GEE, each with its 95% confidence interval, stating what kind of odds ratio each one is. Why is the GLMM odds ratio further from 1?

Model answerThe GLMM gives OR 2.31 (95% CI 1.34 to 3.98): a patient in an urban clinic has about 2.3 times the odds of referral of a patient of the same age and smoking status in a rural clinic with the same baseline, so it is a clinic-specific (conditional) odds ratio. GEE gives OR 2.04 (95% CI 1.16 to 3.59): across the whole population of patients, averaging over clinics, the odds of referral are about twice as high in urban clinics as in rural clinics for patients of the same age and smoking status; it is a population-averaged (marginal) odds ratio. Both intervals exclude 1, so both models show higher odds of referral in urban clinics. The GLMM odds ratio is further from 1 because clinics differ in their baseline odds (clinic variance 0.30 on the log-odds scale): averaging the probability of referral over clinics with high and low baselines pulls the population comparison toward 1, and the weighting used by GEE also contributes.

2. Compare the standard errors for smokerYes and for clinic_urbanurban across the three models. Why does one change much more than the other, and which results should be reported?

Model answerFor urban clinic, the standard error rises from 0.168 in glm() to 0.279 in the GLMM and 0.289 in GEE, while for smoking it stays almost the same (0.188, 0.194 and 0.186). Urban clinic is a clinic-level predictor: it takes one value per clinic, so the 966 patients supply only 30 independent values of it, and glm(), which treats every patient as independent, gives it a standard error, confidence interval and p-value that are too small (overconfident). Smoking varies among patients within every clinic, so its information comes from comparisons within clinics, and allowing for the clinics changes its standard error little. The GLMM (a random intercept for each clinic) and GEE (sandwich standard errors calculated from the variation between clinics) both allow for the clustering, so one of them should be reported, not glm().

3. Run your own code: at the end of the script, type shuffled <- clinics[order(clinics$age), ] and summary(update(gee, data = shuffled)) and run them. What does the Number of clusters line show now, and what happens to the standard error for clinic_urbanurban? Why must the data be sorted before geeglm(), and are 30 clinics enough for GEE? Which odds ratio for urban clinic would you give a clinic manager asking about patients in her own clinic, and which a provincial planner asking about the whole province?

Model answerThe shuffled model reports Number of clusters: 705 instead of 30, because geeglm() treats consecutive rows with the same id as one cluster, so the scattered rows of each clinic were split into many small clusters. Its standard error for urban clinic falls to 0.171, close to the glm() value of 0.168, so the clustering is no longer allowed for, and R gives no warning. Sorting the data by clinic before fitting keeps each clinic's rows together, and the line Number of clusters: 30 in the sorted model confirms that the 30 clinics were read correctly. GEE needs about 30 to 40 clusters or more for reliable sandwich standard errors, so 30 clinics is just enough, at the lower limit; the agreement with the GLMM is reassuring. A clinic manager would get the GLMM (clinic-specific) odds ratio, 2.31, which compares patients within a clinic, and a provincial planner the GEE (population-averaged) odds ratio, 2.04, which describes the association across all patients in the province.
Saved.

Your output from Activity 5.3 shows both clustered models widening the standard error for urban clinics, and the unsorted refit shows why the data must be sorted before GEE is fitted. The key takeaways below draw the section together before its knowledge check.

Key Takeaways

  • 6.1 A GLMM and GEE both extend logistic regression to clustered data, but they allow for the clustering in different ways. A GLMM, fitted with glmer(..., family = binomial), works like the linear mixed model: a random intercept such as (1 | clinic_id) gives each cluster its own baseline log-odds, with the shifts drawn from a normal distribution. GEE, fitted with geeglm(), fits one logistic regression for the whole population with no cluster effects, uses a working correlation (usually "exchangeable", which assumes that any two members of a cluster are equally correlated) to weight the data, and calculates corrected, or sandwich, standard errors from the variation between clusters. You can tell them apart in code because the GLMM puts the cluster inside the formula, while GEE names it in the id argument. If a question asks how each method allows for clustering, the short answer is a random intercept for the GLMM and sandwich standard errors for GEE.
  • 6.2 Odds ratios from both models are read as you read them from ordinary logistic regression: R gives them with exp(fixef(glmm)) or exp(coef(gee)) (2.31 for urban clinics in the GLMM, the exponentiated coefficient of 0.836), and you describe the change in the odds, for example that patients in urban clinics have about twice the odds of referral of patients in rural clinics of the same age and smoking status. Compared with an ordinary glm(), both methods widen the standard error of a cluster-level predictor (0.168 for urban clinic rising to 0.279 in the GLMM and 0.289 in GEE, clearly larger), while the standard error of a person-level predictor such as smoking hardly changes (0.188, 0.194 and 0.186). This is the same pattern you met for blood pressure in Section 1, and it has the same explanation: urban clinic has only 30 independent values, whereas smoking can be compared within every clinic. Whenever you interpret one of these odds ratios, give its confidence interval, check whether the interval includes 1, and say whether the odds ratio is cluster-specific or population-averaged.
  • 6.3 The two models answer slightly different questions, so an exam answer should always name the kind of odds ratio it reports. A GLMM odds ratio is cluster-specific (conditional): it compares two people in the same cluster, or in clusters with the same baseline, so the GLMM's 2.31 compares similar patients in an urban and a rural clinic with the same baseline. A GEE odds ratio is population-averaged (marginal): it compares all urban-clinic patients with all rural-clinic patients of the same age and smoking status, across every clinic. For a binary outcome, averaging over clusters with high and low baselines pulls the population comparison toward 1, so the GEE odds ratio is usually a little closer to 1 (2.04 against 2.31), and the gap grows as the clusters vary more; for a continuous outcome, as in Section 2, the two kinds of estimate are the same. Two template openings help: "for two people in the same cluster..." for a GLMM result and "across the whole population, averaging over clusters..." for a GEE result.
  • 6.4 Choose between the two mainly by asking who wants the answer and what they want to know. A question about individual clusters, such as a clinic manager asking how much higher the odds are for a patient in her own clinic, points to the GLMM and its cluster-specific odds ratio; a question about the population as a whole, such as a provincial planner asking how referral differs across the province, points to GEE and its population-averaged odds ratio. Then check the number of clusters: a GLMM works with about 20 to 30, but GEE needs about 30 to 40 or more for reliable sandwich standard errors, so the 30 clinics here sit at the lower limit for GEE. Keep in mind that only the mixed model estimates how much the clusters vary, while GEE makes no assumption about how the cluster effects are distributed. Many papers report one method as the main analysis and the other as a sensitivity analysis, and when the two agree, as they do for urban clinics here, the conclusion is stronger.
  • 6.5 Each approach has a short checklist, and Table 5.7 sets it out. Both methods need a binary outcome with enough events, about 10 per predictor (check with table(); there were 246 referrals here), and clusters that are independent of one another, which you judge from the design. For the GLMM, check that the cluster effects look roughly normal on a Q-Q plot of ranef(), that there are about 20 to 30 clusters or more, and whether R printed any convergence warnings. For GEE, sort the data by the cluster identifier before fitting, give the right variable to id, and confirm that the Number of clusters line matches the number you expect and is large enough (about 30 to 40 or more). Sorting is the easy step to forget: geeglm() treats consecutive rows with the same id as one cluster, so scattered rows split a cluster and give wrong standard errors without any warning.
Knowledge check: this section

1. A study of influenza vaccination (yes or no) includes 1,920 adults aged 65 and older registered at 48 primary care clinics (40 per clinic), and 24 of the clinics use an automated reminder system. Vaccination was modelled with ordinary logistic regression, a GLMM and GEE, each adjusted for age, giving the odds ratios below (standard errors on the log-odds scale in brackets). Which statement explains the pattern of standard errors and says which results to report?

ModelChronic condition: OR (SE)Reminder system: OR (SE)
glm()1.70 (0.10)1.30 (0.10)
GLMM, glmer()1.80 (0.11)1.36 (0.19)
GEE, geeglm()1.65 (0.10)1.26 (0.18)
The reminder system takes one value per clinic, so glm(), which treats adults as independent, overstates its information. The GLMM allows for clustering with clinic intercepts and GEE with sandwich standard errors, so either is reported.
All three models use the same 1,920 adults. The GLMM and GEE standard errors are larger because they allow for clustering; glm() only looks more precise because it wrongly assumes independence.
This swaps the two methods: the GLMM adds a random intercept for each clinic and GEE uses sandwich standard errors. Neither widens every standard error equally; the change is large for the clinic-level predictor and small for chronic condition.
Both methods allow for clustering for every predictor. Chronic condition varies within every clinic and can be compared within clinics, so allowing for clustering changes its standard error little.
Correct answer: the reminder system is a clinic-level predictor, and the GLMM or GEE result should be reported. The information needed is the level of each predictor and how each method allows for clustering: the standard error for the reminder system rises from 0.10 in glm() to 0.19 in the GLMM and 0.18 in GEE, while the standard error for chronic condition stays at about 0.10, because chronic condition varies within every clinic and can be compared within clinics. The reminder system takes one value per clinic, so glm(), which counts 1,920 independent adults, overstates the information about it. The option preferring glm() is wrong because all three models use the same 1,920 adults; glm() looks more precise only because it wrongly assumes independence. (Section 3, 6.1 How a GLMM and GEE Allow for Clustering and 6.2 Odds Ratios and Standard Errors from the Three Models, Table 5.6; Objectives 6.1 and 6.2)

2. In the vaccination study, the GLMM (glmer() with a random intercept for each clinic) gives an odds ratio of 1.80 (95% CI 1.45 to 2.23) for having a chronic condition, adjusted for age and the reminder system. Which interpretation is correct?

This describes a population-averaged comparison across all adults, which is what GEE estimates, and it reads an odds ratio as a ratio of probabilities. A GLMM odds ratio compares odds for adults who share a clinic.
An odds ratio compares the odds of vaccination between two groups; it is not the proportion vaccinated. The value 1.80 means about 80% higher odds, not that 80% were vaccinated.
A GLMM gives each clinic its own baseline log-odds, so its odds ratio compares adults in the same clinic or in clinics with the same baseline: a cluster-specific, or conditional, odds ratio.
Chronic condition is measured on each adult, and the GLMM compares adults within clinics. The odds ratio describes a difference between individuals, not a comparison between clinics with different case mixes.
Correct answer: a cluster-specific odds ratio of about 1.8 for two adults who share a clinic. The information needed is the odds ratio, its interval and the model that produced it: a GLMM gives each clinic its own baseline log-odds, so its odds ratios compare adults in the same clinic (or in clinics with the same baseline). The odds of vaccination are about 80% higher (OR 1.80), and the 95% CI (1.45 to 2.23) excludes 1. The first option is tempting, but it describes a population-averaged comparison, which is what GEE estimates, and it reads the odds ratio as a ratio of probabilities. (Section 3, Equation 5.4 and 6.3 Clinic-Specific and Population-Averaged Odds Ratios, including the flip card “Clinic-Specific Odds Ratio”; Objectives 6.2 and 6.3)

3. GEE (geeglm() with the clinic as the cluster) gives an odds ratio of 1.65 (95% CI 1.36 to 2.01) for chronic condition, compared with 1.80 from the GLMM. Which statement is correct?

GEE fits a model without clinic effects, so its odds ratio is population-averaged rather than a within-clinic comparison. It does allow for clustering through sandwich standard errors; it is closer to 1 because of averaging over clinics.
GEE estimates a population-averaged odds ratio. For a binary outcome, mixing clinics with high and low baselines pulls the population comparison toward 1, so it is usually a little closer to 1 than the cluster-specific odds ratio.
Neither model is wrong. The GLMM and GEE answer different questions, within-clinic and population-wide, and for a binary outcome the population-averaged odds ratio is expected to be closer to 1.
The two odds ratios are expected to be identical only for a continuous outcome. For a binary outcome, cluster-specific and population-averaged odds ratios differ, and the gap grows with the variation between clinics.
Correct answer: a population-averaged odds ratio of about 1.65, a little closer to 1 than the GLMM odds ratio. The information needed is that GEE fits a model with no clinic effects, so its odds ratio compares groups across the whole population (population-averaged, or marginal). For a binary outcome, mixing clinics with high and low baselines pulls the population comparison toward 1, so the population-averaged odds ratio is usually a little closer to 1 than the cluster-specific one, and the gap grows with the variation between clinics. Both odds ratios answer valid but different questions, so neither model is wrong, and identical estimates are expected only for a continuous outcome. (Section 3, 6.3 Clinic-Specific and Population-Averaged Odds Ratios and Worked Example 5.3; Objective 6.3)

4. Two people ask about the vaccination study, which has 48 clinics. A clinic manager wants to know how much higher the odds of vaccination are for a patient with a chronic condition than for a similar patient in her own clinic. A provincial planner wants to know how vaccination differs between older adults with and without a chronic condition across the province. Which estimates suit them?

This reverses the two estimates. GEE fits one model without clinic effects and describes the population, while the GLMM, with a baseline for each clinic, describes comparisons within a clinic.
glm() treats the adults as independent, so its standard errors are too small, and it answers neither question as well as a method that allows for clustering. Using all adults without correction is a weakness, not a strength.
GEE needs more clusters than the GLMM, about 30 to 40 or more, not fewer. GEE also gives a population-averaged estimate, which does not suit the manager's question about patients within her clinic.
The choice follows the question: a within-clinic comparison needs a cluster-specific estimate from the GLMM, and a population question needs a population-averaged estimate from GEE, which has enough clusters here.
Correct answer: the GLMM odds ratio for the manager and the GEE odds ratio for the planner. The information needed is who is asking and about what: a question about patients within one clinic calls for a cluster-specific estimate (GLMM), and a question about the population calls for a population-averaged estimate (GEE). The number of clusters matters too: GEE needs about 30 to 40 clusters or more for reliable sandwich standard errors, so 48 clinics is enough, while with fewer clusters the GLMM would be the safer choice. Reversing the two is the common mistake, and GEE needs more clusters than the GLMM, not fewer. (Section 3, 6.4 Choosing Between a GLMM and GEE; Objective 6.4)

5. Which of the following is a check that the GEE result for the vaccination study requires before it is reported?

The Q-Q plot of cluster effects is a GLMM check. GEE has no clinic effects and makes no assumption about their distribution, so there is nothing of this kind to plot.
A small working correlation still matters when clusters are large, just as a small ICC does. A low alpha is not a reason to switch to glm(), and no threshold of 0.5 is required.
geeglm() treats consecutive rows with the same id as one cluster, so unsorted data split clinics and give wrong standard errors without a warning. Sorting and confirming the reported number of clusters prevents this.
isSingular() is a check for mixed models, not GEE, and TRUE would mean a variance was estimated at or near zero. GEE estimates no clinic variance, so this check does not apply.
Correct answer: data sorted by clinic and a Number of clusters line showing 48. The information needed is how geeglm() identifies clusters: it treats consecutive rows with the same id as one cluster, so unsorted data split a clinic into several clusters and give wrong standard errors without any warning. The Q-Q plot of cluster effects is a GLMM check, because GEE has no clinic effects and makes no assumption about their shape, and a small working correlation still matters when clusters are large, just as a small ICC does (Section 1, Worked Example 5.1), so a low alpha does not justify glm(). The other requirements for both methods are a binary outcome with enough events (about 10 per predictor), independent clusters and, for GEE, about 30 to 40 clusters or more. (Section 3, 6.5 Checking a GLMM and GEE, Table 5.7 and the callout “Sort the Data Before Using geeglm()”; Objective 6.5)

6. A study of a smoking-cessation counselling service, which 24 of 48 community pharmacies offer, modelled whether each of 2,400 customers had quit at six months, adjusted for age and heavy smoking. The odds ratios for the service are shown below (the GEE output reports Number of clusters: 48). The provincial health ministry asks how the odds of quitting compare, across all customers in the province, between pharmacies with and without the service. Which result and reading suit the ministry?

ModelCounselling service: OR (95% CI)
glm()1.55 (1.30 to 1.85)
GLMM, glmer()1.62 (1.15 to 2.28)
GEE, geeglm()1.50 (1.08 to 2.08)
The GLMM odds ratio compares customers of pharmacies with the same baseline, which suits a question about one pharmacy. The ministry asks about all customers in the province, a population-averaged question that GEE answers.
The ministry's question is about all customers, which is the population-averaged comparison GEE estimates; with 48 pharmacies there are enough clusters for its sandwich standard errors.
The glm() interval is narrow only because it treats 2,400 customers as independent, although the service takes one value per pharmacy. A narrow interval built on a false assumption is overconfident, not precise.
The model is right but the reading is wrong. An odds ratio of 1.50 means 50% higher odds of quitting, not that 50% of customers quit; an odds ratio is not a proportion.
Correct answer: the GEE (population-averaged) odds ratio of 1.50. The information needed is who is asking and what each model compares: the ministry asks about all customers in the province, which is the population-averaged comparison that GEE estimates, while the GLMM's odds ratio compares customers of pharmacies with the same baseline, which suits a question about one pharmacy. GEE needs about 30 to 40 clusters or more for reliable sandwich standard errors, so 48 pharmacies are enough. The glm() interval is narrow only because it treats the 2,400 customers as independent, although the service takes one value per pharmacy, and an odds ratio of 1.50 means 50% higher odds, not that 50% of customers quit. (Section 3, 6.3 Clinic-Specific and Population-Averaged Odds Ratios with the flip card “Population-Averaged Odds Ratio”, 6.4 Choosing Between a GLMM and GEE and Table 5.7; Objectives 6.3 and 6.4)
● Complete the knowledge check to continue.
Section 4 of 4

Repeated Measures Over Time

⏱ Estimated time: 45 minutes
Lesson 5 · Section 4

Repeated Measures Over Time

When the same people are measured several times, each person becomes a cluster of visits.

Running example

A wellness trial with four visits

200
adults, 100 per arm
4
visits: 0, 6, 12 and 18 months
2
outcomes: blood pressure and adherence

Each person is a cluster of visits, so the visits from one person are not independent.

Data layout

Long format: one row per visit

idarmvisit (months)sbp_mmhgadherent
R0001control01320
R0001control61310
R0001control121410
R0001control181260
R0002control01271

Mixed models and GEE need long format, with an id column that identifies the person (the cluster).

Look first

Individual lines and arm averages

Panel A: blood pressure over four visits for 24 participants, one line each. Panel B: mean blood pressure by arm, falling from 130.4 to 127.0 in the control arm and from 128.9 to 123.1 in the intervention arm.
Each person stays in their own band (A), and the intervention arm falls more steeply (B).
Missed visits

Mixed models use every visit that was attended

96
of 800 measurements missing (12%)
121
of 200 people attended all four visits
704
measurements used by the mixed model

The results are valid if missing a visit depends only on information in the model (missing at random).

The model

Does blood pressure fall faster in the intervention arm?

lmer(sbp_mmhg ~ arm * visit + (1 | id), data = visits)

TermWhat it measures
(1 | id)A baseline level for each person (random intercept)
arminterventionThe difference between the arms at the start (month 0)
visitThe change per month in the control arm
armintervention:visitThe extra change per month in the intervention arm
Reading the output

An extra fall of about 2.7 mmHg over 18 months

TermEstimateSEp-value
Intervention arm at month 0−1.32 mmHg1.070.22
Change per month, control arm−0.160 mmHg0.043< 0.001
Extra change per month, intervention arm−0.149 mmHg0.0610.014
−2.7
mmHg extra over 18 months (18 × −0.149)
0.56
ICC: how alike each person's visits are
A repeated binary outcome

Adherence over time

Proportion adherent at months 0, 6, 12 and 18. Control: 0.43, 0.41, 0.40, 0.37. Intervention: 0.42, 0.55, 0.69, 0.73.
Adherence rises in the intervention arm and drifts down in the control arm, and the model is fitted with geeglm(adherent ~ arm * visit, id = id, family = binomial, corstr = "exchangeable").
Reading the output

Adherence rises faster in the intervention arm

TermOdds ratio95% CI
Intervention arm at month 01.000.60 to 1.69
Change per month, control arm0.990.96 to 1.02
Extra change per month, intervention arm1.0931.041 to 1.147

Over one six-month interval, 1.0936 ≈ 1.7. The working correlation between a person's visits is 0.07, and there are 200 clusters.

Assumptions and checks

What to check in a repeated-measures analysis

CheckHow
Long format with a person identifierhead(data); sort by id and visit for GEE
Time coded and shaped correctlyPlot the averages by visit; straight lines support a linear trend
Residuals for the mixed modelThe residual and Q-Q plots from Section 2
Missed visitsCount them by visit and compare people who missed visits with those who did not
Lesson summary

Data structure, model and main check

SituationModel and R functionMain check
Any clustered dataICC and design effectHow strong is the clustering, and how much information is lost?
Continuous outcomeLinear mixed model, lmer()Residual plots, Q-Q of cluster effects, number of clusters
Binary outcome, cluster-specificGLMM, glmer()Events, convergence warnings
Binary outcome, population-averagedGEE, geeglm()About 30 to 40 clusters or more, sorted data
Repeated measuresThe same models with the person as the clusterLong format, time trend, missed visits

Overview

This section applies the methods of the lesson to repeated measures, in which the same people are measured at several times. The example is phaa_repeated.csv, a simulated wellness trial in which 200 adults were assigned to a control arm or an intervention arm (100 each) and were due to attend visits at 0, 6, 12 and 18 months. At each visit the study recorded systolic blood pressure (sbp_mmhg) and whether the person was adhering to the program (adherent = 1). The section explains why each person is a cluster of visits, how repeated-measures data are laid out, how missed visits affect the analysis, how a linear mixed model with an arm-by-time interaction answers the main question of the trial, and how GEE models the repeated binary outcome.

Learning Objectives

  • 1.2 Recognise repeated-measures data and explain why each person is a cluster of observations.
  • 7.1 Distinguish long from wide format and identify the variables a repeated-measures model needs.
  • 7.2 Explain how a complete-case analysis, a mixed model and GEE each handle missed visits, and what the missing-at-random condition means.
  • 7.3 Interpret a linear mixed model with an interaction between study arm and time.
  • 7.4 Interpret a GEE model for a repeated binary outcome, and list the checks for repeated-measures analyses.

1.2 Repeated Measures as Clustered Data

When the same person is measured more than once, the measurements share that person's genes, habits, circumstances and baseline health, so they are more alike than measurements from different people. Each person is therefore a cluster of visits, identified by a variable such as id, and the methods from the earlier sections apply with the person in place of the clinic; the cluster is the person, not the visit or the year. In this trial, the ICC for blood pressure is 0.56: more than half of the variation is a stable difference between people, much more than the clustering of patients in clinics (0.175), because repeated measurements of one person share far more than patients who only share a clinic. Even a weak correlation between a person's visits still breaks the independence assumption, so an ordinary regression is not the right choice for repeated measures.

A plot of the measurements over time shows this dependence before any model is fitted. Figure 5.7 shows the blood pressure of individual participants at each visit and the average in each arm.

Panel A: systolic blood pressure at four visits for 24 participants, one line each, coloured by arm. Panel B: mean blood pressure by arm at months 0, 6, 12 and 18, falling from 130.4 to 127.0 in the control arm and from 128.9 to 123.1 in the intervention arm.
Figure 5.7. Panel A shows one line for each of 24 randomly chosen participants, and panel B shows the average blood pressure in each arm at each visit. Each person tends to stay in their own band, and the intervention arm falls more steeply than the control arm.

The spaghetti plot (panel A) shows the dependence directly: people start at very different levels and tend to stay there. The averages (panel B) fall in both arms, from 130.4 to 127.0 mmHg in the control arm and from 128.9 to 123.1 mmHg in the intervention arm, and the roughly straight lines suggest that a linear trend over time is reasonable. To fit a model to these repeated measurements, the data first have to be arranged in the right layout.

7.1 Long and Wide Format

Repeated-measures data can be stored in wide format, with one row per person and a column for each visit, or in long format, with one row per visit and columns for the person identifier, the visit time and the outcome. Mixed models and GEE in R need long format, because they follow change over time and need the person identifier and the time on every row. The trial file is already in long format, with 800 rows, four for each of the 200 people. Table 5.8 shows the first five rows of the file.

Table 5.8. The first five rows of phaa_repeated.csv in long format, with one row per visit.

idarmvisit (months)sbp_mmhgadherent
R0001control01320
R0001control61310
R0001control121410
R0001control181260
R0002control01271

The id column identifies the person (the cluster), visit records months since the start, and each person appears once for each visit. In wide format, the same person R0001 would be a single row with columns such as sbp_0, sbp_6, sbp_12 and sbp_18. The base R function reshape() or tidyr::pivot_longer() converts wide data to long format. A long file also shows a missed visit simply as a row with a missing outcome, which raises the question of how the analysis should treat missed visits.

7.2 Missed Visits

Of the 800 planned measurements of blood pressure, 96 are missing (the first table() line of Activity 5.4 counts them by visit), and only 121 of the 200 participants attended all four visits. One simple option would be to drop everyone with a missed visit, which is called a complete-case analysis. This would discard 79 people, and it causes two problems. The first is a loss of precision, because the measurements those 79 people did provide are thrown away. The second, more serious, problem is possible bias. If the people who missed visits differ from those who attended every one, for example because people whose blood pressure was rising were more likely to drop out, then the people who remain are no longer typical of the trial, and the results describe only the more committed or healthier participants.

There is one situation in which a complete-case analysis of a regression model still gives unbiased coefficients: when the chance that a record is incomplete depends only on the predictors in the model and not on the outcome. In that case dropping incomplete records costs precision but does not distort the coefficients; this exception refines the general statement that listwise deletion is biased unless the data are MCAR. In repeated-measures data, however, the exception rarely helps, because a missed visit often depends on the person’s earlier outcome values. For example, a participant whose blood pressure was high at 6 months may be less likely to attend at 12 months, and then the exception does not apply. A mixed model avoids discarding anyone. Because it models each visit as a separate row, it uses every measurement that was taken, here 704 measurements from all 200 people, so a person who attended three of the four visits still contributes those three measurements.

Whether the mixed model's results are valid depends on why visits were missed. The recall below revisits the three missing-data mechanisms from Lesson 2; try its retrieval question before opening the answer.

Recall: Missing-Data Mechanisms (Lesson 2, Section 1)

Lesson 2, Section 1 set out Rubin’s three missing-data mechanisms. Data are missing completely at random (MCAR) when the chance that a value is missing is unrelated to any value, observed or not. They are missing at random (MAR) when that chance depends only on values that were observed, and missing not at random (MNAR) when it depends on the missing value itself. Students who took HSCI 230 first met these mechanisms in Lesson 8, Section 2 (Attrition and Nonresponse Bias) and Lesson 11, Section 2 (Statistical Inference and Model Issues).

Retrieval question. A participant in the wellness trial misses the 12-month visit. Give one reason for the missed visit that would make the missing blood pressure MAR, and one that would make it MNAR.

AnswerThe value is MAR if the chance of missing the visit depends only on recorded information, for example if people whose blood pressure was high at the 6-month visit were more likely to miss the next one. It is MNAR if the chance depends on the unrecorded value itself, for example if the person stayed away because their blood pressure was high at the time of the 12-month visit, in a way that the earlier visits could not predict.

The mechanism that matters for a mixed model is missing at random, stated in plain terms for the trial in the definition below.

Missing at Random, in Plain Terms

A mixed model fitted to all available visits gives valid results when the chance of missing a visit depends only on information the model already uses, such as the person's arm or their blood pressure at earlier visits. This condition is called missing at random (MAR). It would fail if, for example, people stayed away from a visit because their blood pressure was high on that day, in a way the earlier visits could not predict. MAR cannot be proved from the data, but it is a much weaker assumption than the complete-case assumption that people who attended every visit are typical of everyone. Comparing people who missed visits with those who did not at their first visit shows which measured characteristics predict a missed visit, so that they can be included in the model. The comparison cannot confirm MAR, because MAR concerns the values that were never recorded.

GEE needs a stronger condition about missed visits than the mixed model. Its results are valid when the chance of missing a visit depends only on the predictors in the GEE model, here arm and visit. The difference arises because the mixed model treats a person’s visits as related to one another, so the visits a person did attend inform the model about the visits that person missed. GEE, by contrast, does not use earlier outcomes in this way, so it cannot correct for dropout that depends on them. For example, if participants whose blood pressure was high at 6 months were more likely to miss the 12-month visit, the mixed model can allow for this through their 6-month value, whereas standard GEE cannot. If missed visits depend on earlier outcomes, such as earlier blood pressure or adherence, a mixed model, or a weighted form of GEE that is beyond this lesson, is preferred.

When the missing-at-random condition holds, a mixed model fitted to all 704 available measurements gives valid results. The next part fits that model to the blood pressure data.

7.3 A Mixed Model for Change Over Time

The model for blood pressure includes a random intercept for each person and fixed effects for arm, visit and their interaction. Equation 5.5 writes the model; the interaction term β3 carries the main question of the trial.

Random-intercept model for change over time
\[ \text{SBP}_{it} = \color{#6D28D9}{\beta_0} + \color{#C2410C}{u_i} + \color{#1D4ED8}{\beta_1}\,\text{arm}_i + \color{#1D4ED8}{\beta_2}\,\text{visit}_{t} + \color{#0B7B6B}{\beta_3}\,(\text{arm}_i \times \text{visit}_{t}) + \varepsilon_{it} \]Eq 5.5
Blood pressure for person i at time t equals an overall intercept, a shift for person i, a difference between arms at the start and a change per month in the control arm, plus the extra change per month in the intervention arm (β3), which is the effect of the program on the trend.

Table 5.9 gives the estimates of the fixed effects for arm, visit and their interaction, with an interpretation of each. The arm term describes only the difference between the arms at the start, so reading it as the effect of the program is a common error. This is because, once the model includes an interaction between arm and visit, the arm coefficient refers to the point where visit equals 0, that is, month 0, before the program could have had any effect. The visit term, in turn, is the change per month in the control arm only. The effect of the program on the trend is the interaction, which measures how much faster or slower blood pressure changes each month in the intervention arm than in the control arm. In other words, the trial's main question, whether blood pressure falls faster with the program, is answered by the interaction term and not by the arm term.

Table 5.9. Fixed effects from the random-intercept model for blood pressure over time.

TermEstimateSEp-valueInterpretation
Intervention arm (month 0)−1.32 mmHg1.070.22The arms start at similar levels, as expected after randomization.
Visit (control arm, per month)−0.160 mmHg0.043< 0.001Blood pressure falls by about 0.16 mmHg per month in the control arm.
Arm × visit (per month)−0.149 mmHg0.0610.014Blood pressure falls an extra 0.15 mmHg per month in the intervention arm.

Activity 5.4 obtains the extra fall over the 18 months of the trial in R with 18 * fixef(lmm_t)[["armintervention:visit"]], which prints −2.69: blood pressure in the intervention arm falls about 2.7 mmHg more than in the control arm over the trial. The random effects give a between-person variance of 34.80 and a within-person variance of 27.58, and the ICC line in the activity prints 0.56, the between-person share of their total.

Worked Example 5.4 applies Equation 5.5 to predict blood pressure at month 18 in each arm from the intercept and the estimates in Table 5.9; watch which terms each prediction adds.

Worked Example 5.4: Predicted Blood Pressure at Month 18

For a typical person in each arm at month 18, predict() in R gives 127.1 mmHg in the control arm and 123.1 mmHg in the intervention arm. The control-arm prediction combines the intercept with 18 months of the control-arm change per month; the intervention-arm prediction also adds the difference at the start and 18 months of the extra change per month, which is why the two arms end further apart than they began. Both are close to the observed averages of 127.0 and 123.1 mmHg.

The predictions show how the per-month estimates add up over the trial. Because the arms of this trial were assigned at random, a difference in trends between them can be read as an effect of the program. When the groups were not randomized, for example when clinics or workplaces chose whether to run a program, the same interaction describes how the trends differ, but other differences between the groups could explain part of it, so it is reported as an association.

The model can be extended in two ways that appear in published longitudinal studies but are beyond the scope of this lesson. Expand the panel below to read about random slopes for time and a correlation that depends on the time between visits.

Going further: random slopes for time and correlation over time

The random-intercept model gives each person their own level but the same slope over time within each arm. A random slope, (visit | id), would let each person have their own rate of change. In this trial that model produces a convergence warning, because four visits per person give little information about individual slopes. Visits close together in time are also often more alike than visits far apart, a pattern that models with an autoregressive correlation structure (such as AR(1)) describe. Both extensions appear in published longitudinal studies and are beyond the scope of this lesson.

The mixed model shows an extra fall of about 2.7 mmHg over 18 months in the intervention arm. The trial also recorded a binary outcome at each visit, and the next part models it with GEE.

7.4 GEE for a Repeated Binary Outcome, and the Checks for Repeated Measures

Adherence is recorded as yes or no at each visit. The share adherent falls slightly in the control arm (0.43 at month 0 to 0.37 at month 18) and rises steadily in the intervention arm (0.42 to 0.73). Figure 5.8 shows these proportions at each visit.

Proportion adherent by arm at months 0, 6, 12 and 18. Control: 0.43, 0.41, 0.40, 0.37. Intervention: 0.42, 0.55, 0.69, 0.73.
Figure 5.8. The lines show the proportion of participants adhering to the program at each visit in each arm. Adherence drifts down in the control arm and rises in the intervention arm.

A GEE model with the person as the cluster (id = id) gives population-averaged odds ratios for the trial, which matches the question of how the program changes adherence across participants. With 200 people there are plenty of clusters for the corrected standard errors. Table 5.10 gives the odds ratios from this model.

Table 5.10. Odds ratios from the GEE model for adherence over time.

TermOdds ratio95% CIInterpretation
Intervention arm (month 0)1.000.60 to 1.69The arms start with the same odds of adherence.
Visit (control arm, per month)0.990.96 to 1.02Adherence in the control arm barely changes.
Arm × visit (per month)1.0931.041 to 1.147The monthly odds ratio in the intervention arm, which R prints as 1.079 from exp(coef(gee_a)[["visit"]] + coef(gee_a)[["armintervention:visit"]]), shows a rise of about 8% per month.

Over one six-month interval between visits, R gives the interaction odds ratio with exp(6 * coef(gee_a)[["armintervention:visit"]]), which prints 1.70. Odds ratios multiply over time rather than add, so the interaction odds ratio for six months is not six times the monthly one. This is because the model is linear on the log-odds scale: each month adds the same amount to the log-odds, and adding on the log scale corresponds to multiplying on the odds scale. A monthly odds ratio is therefore applied once for each month, so its effect compounds over six months in the way that interest compounds in a savings account, which is why R calculates the six-month value by exponentiating six times the monthly coefficient. The estimated exchangeable correlation between a person's visits is 0.07, much weaker than the correlation for blood pressure, but GEE allows for it in the standard errors. Provided that missed visits depend only on arm and visit, GEE therefore shows a steady rise in adherence in the intervention arm.

Before either repeated-measures model is reported, it needs some of the checks from the earlier sections together with checks of its own, which concern the layout of the data, the coding of time and the missed visits. Table 5.11 lists them for the mixed model and for GEE.

Table 5.11. Assumptions and requirements of a repeated-measures analysis, with how to check them.

Assumption or requirementHow to check it
Data in long format with a person identifierhead(data); for GEE, sort by id and then by visit.
Time coded correctly and the trend shape reasonablePlot the averages by visit; roughly straight lines support a linear trend in visit.
Residual assumptions for the mixed modelResiduals versus fitted values and Q-Q plots, as in Section 2.
Missed visits: missing at random for the mixed model; related only to the model's predictors for GEECount missing values by visit and compare people who missed visits with those who did not. This shows which variables to include; it cannot confirm MAR.
Enough clusters for GEEThe Number of clusters line of the output (200 here).
Independent peopleJudged from the study design (one person per household, for example).

These checks complete the analysis of the wellness trial and the methods of the lesson.

Additional Context

This closing review adds no new methods; it sets the four examples of the lesson side by side. The four sections apply one approach. The cluster variable is identified first, the clustering is measured with the ICC, and a model is chosen that allows for it: a linear mixed model for a continuous outcome, a GLMM or GEE for a binary outcome, and the same models with the person as the cluster for repeated measures. Table 5.12 summarises the four examples with the cluster, the model and R function, and the main result of each.

Table 5.12. The four examples of the lesson, with the cluster, model and main result of each.

ExampleClusterModel and R functionMain result
Blood pressure in clinicsClinicLinear mixed model, lmer()Urban minus rural 1.85 mmHg (−1.85 to 5.54); ICC 0.24
Referral in clinicsClinicGLMM, glmer(), and GEE, geeglm()Urban OR 2.31 (GLMM) and 2.04 (GEE)
Blood pressure over timePersonLinear mixed model, lmer()Extra fall of 0.149 mmHg per month in the intervention arm
Adherence over timePersonGEE, geeglm()Interaction OR 1.093 per month (1.041 to 1.147)

Activity 5.4 lets you fit the mixed model for blood pressure over time and then the GEE model for adherence in R, from the check of missed visits to the interaction over the whole trial: run the script and answer the questions from your own output.

R Activity 5.4: repeated measures with a mixed model and GEE

Text version of the activity: the code of the run-along script, its console output and notes on reading it.

library(lmerTest)   # loads lme4 and adds p-values to lmer() output
visits <- read.csv("https://heal-learn.ca/r-activities/data/phaa_repeated.csv")   # course data, read from the website
visits$id  <- factor(visits$id)   # the person: the cluster
visits$arm <- factor(visits$arm, levels = c("control", "intervention"))
head(visits, 8)              # long format: one row per visit
Console output
id arm age female visit sbp_mmhg adherent 1 R0001 control 54 1 0 132 0 2 R0001 control 54 1 6 131 0 3 R0001 control 54 1 12 141 0 4 R0001 control 54 1 18 126 0 5 R0002 control 59 1 0 127 1 6 R0002 control 59 1 6 125 0 7 R0002 control 59 1 12 131 1 8 R0002 control 59 1 18 143 1

The first eight rows show the long format: the first person has four rows, one for each visit at months 0, 6, 12 and 18, followed by the second person. The id column identifies the person, who is the cluster, and visit records months since the start.

table(visit = visits$visit, missing = is.na(visits$sbp_mmhg))
all4 <- tapply(!is.na(visits$sbp_mmhg), visits$id, all)   # all four visits?
table(all4)                  # FALSE: missed at least one visit
first <- visits[visits$visit == 0, ]           # each person's month-0 row
round(tapply(first$sbp_mmhg, all4[first$id], mean, na.rm = TRUE), 1)
Console output
missing visit FALSE TRUE 0 180 20 6 176 24 12 180 20 18 168 32 all4 FALSE TRUE 79 121 FALSE TRUE 129.0 129.9

Between 20 and 32 blood pressures are missing at each visit (96 of the 800 planned measurements), and 121 of the 200 people attended all four visits. People who missed at least one visit had a similar average blood pressure at month 0 (129.0 mmHg) to people who attended every visit (129.9 mmHg). This comparison shows which measured characteristics predict a missed visit; it cannot confirm that the visits are missing at random, because that concerns values that were never recorded.

round(tapply(visits$sbp_mmhg, list(visits$arm, visits$visit), mean, na.rm = TRUE), 1)
lmm_t <- lmer(sbp_mmhg ~ arm * visit + (1 | id), data = visits)
summary(lmm_t)
Console output
0 6 12 18 control 130.4 128.9 128.1 127.0 intervention 128.9 126.8 125.4 123.1 Linear mixed model fit by REML. t-tests use Satterthwaite's method [ lmerModLmerTest] Formula: sbp_mmhg ~ arm * visit + (1 | id) Data: visits REML criterion at convergence: 4672.1 Scaled residuals: Min 1Q Median 3Q Max -2.7904 -0.5636 0.0015 0.5875 3.3301 Random effects: Groups Name Variance Std.Dev. id (Intercept) 34.80 5.899 Residual 27.58 5.252 Number of obs: 704, groups: id, 200 Fixed effects: Estimate Std. Error df t value Pr(>|t|) (Intercept) 130.02227 0.75761 335.39051 171.622 < 2e-16 *** armintervention -1.31505 1.06859 332.64253 -1.231 0.219329 visit -0.15981 0.04314 514.38871 -3.704 0.000235 *** armintervention:visit -0.14925 0.06055 514.62646 -2.465 0.014032 * --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Correlation of Fixed Effects: (Intr) armntr visit armintrvntn -0.709 visit -0.502 0.356 armntrvntn: 0.358 -0.498 -0.712

Reading the output. The averages by arm and visit match Figure 5.7 and fall in roughly straight lines. The line Number of obs: 704, groups: id, 200 confirms that the model used all 704 available measurements from all 200 people. In the Fixed effects block, armintervention (−1.32, p = 0.22) is the difference between the arms at month 0, visit (−0.160, p < 0.001) is the change per month in the control arm, and armintervention:visit (−0.149, p = 0.014) is the extra change per month in the intervention arm.

vc <- as.data.frame(VarCorr(lmm_t))
vc$vcov[1] / sum(vc$vcov)                      # ICC: how alike a person's visits are
18 * fixef(lmm_t)[["armintervention:visit"]]   # extra change over 18 months
Console output
[1] 0.5578058 [1] -2.686584

The first line prints the ICC, 0.56: more than half of the variation in blood pressure lies between people, so a person's visits are strongly alike. The second line prints the extra change in the intervention arm over the 18 months of the trial, −2.69: blood pressure fell about 2.7 mmHg more in the intervention arm than in the control arm.

library(geepack)
adh <- visits[!is.na(visits$adherent), ]       # visits with adherence recorded
adh <- adh[order(adh$id, adh$visit), ]         # each person's rows together
round(tapply(adh$adherent, list(adh$arm, adh$visit), mean), 2)   # share adherent
gee_a <- geeglm(adherent ~ arm * visit, id = id, family = binomial,
                corstr = "exchangeable", data = adh)
summary(gee_a)
options(digits = 7)   # summary() lowers the printed digits; this restores them
round(exp(cbind(OR = coef(gee_a), confint.default(gee_a))), 3)   # odds ratios, 95% CIs
Console output
0 6 12 18 control 0.43 0.41 0.40 0.37 intervention 0.42 0.55 0.69 0.73 Call: geeglm(formula = adherent ~ arm * visit, family = binomial, data = adh, id = id, corstr = "exchangeable") Coefficients: Estimate Std.err Wald Pr(>|W|) (Intercept) -0.278541 0.198606 1.967 0.160773 armintervention 0.005248 0.264926 0.000 0.984197 visit -0.012340 0.016809 0.539 0.462865 armintervention:visit 0.088695 0.024784 12.807 0.000345 *** --- Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 Correlation structure = exchangeable Estimated Scale Parameters: Estimate Std.err (Intercept) 1 0.0292 Link = identity Estimated Correlation Parameters: Estimate Std.err alpha 0.06572 0.03557 Number of clusters: 200 Maximum cluster size: 4 OR 2.5 % 97.5 % (Intercept) 0.757 0.513 1.117 armintervention 1.005 0.598 1.690 visit 0.988 0.956 1.021 armintervention:visit 1.093 1.041 1.147

Reading the output. The shares adherent match Figure 5.8. geeglm() treats consecutive rows with the same id as one cluster, so the rows are sorted by person and visit before the model is fitted. In the GEE output, armintervention (0.005) shows no difference between the arms at month 0, visit (−0.012) shows little change in the control arm, and armintervention:visit (0.089, p < 0.001) shows that adherence rises faster in the intervention arm. The estimated correlation between a person's visits (alpha) is 0.07, and Number of clusters: 200 confirms that each person is a cluster. The odds ratio for the interaction is 1.093 per month (95% CI 1.041 to 1.147); confint.default() gives Wald confidence intervals from the corrected standard errors.

Files for this activity Data Code Answer key

R Reflect on what you just ran

Answer each question from your own output, as on the final exam: find the number, say what it means and state what the model assumes.

1. Only 704 of the 800 planned measurements were taken (Number of obs: 704, groups: id, 200), and 121 of the 200 people attended all four visits. Why is the mixed model, which uses every visit that took place, better than analysing only the 121 people who attended all four? What must be true about the missed visits for its results to be valid?

Model answerAnalysing only the 121 people who attended all four visits would drop the 79 people who missed a visit, together with every visit they did attend, so it would lose information; the mixed model uses all 704 measurements from all 200 people. A complete-case analysis could also be biased if the people who missed visits differ from those who attended every one; here the people who missed a visit had a similar blood pressure at month 0 (129.0 against 129.9 mmHg), so their starting blood pressure does not appear to predict a missed visit. The mixed model gives valid results if the visits are missing at random: whether a visit is missed depends only on information in the model, such as the arm or the person's earlier blood pressure readings, and not on the unrecorded blood pressure itself. This cannot be proved from the data, because it concerns values that were never recorded, so it is stated as an assumption.

2. Interpret the ICC of 0.56 (vc$vcov[1] / sum(vc$vcov)). Which coefficient answers the question whether blood pressure falls faster in the intervention arm? Interpret it, using the change over the 18 months of the trial.

Model answerThe ICC of 0.56 means that more than half of the variation in blood pressure is a stable difference between people, so a person's visits are strongly alike, much more than patients who share a clinic (ICC 0.175). The interaction armintervention:visit answers the question: it is the extra change per month in the intervention arm. Blood pressure falls by a further 0.149 mmHg per month in the intervention arm (SE 0.061, p = 0.014), on top of the fall of about 0.16 mmHg per month in the control arm. Line 28 prints the extra change over the 18 months of the trial, −2.69: blood pressure fell about 2.7 mmHg more in the intervention arm than in the control arm. The model predicts a fall of about 2.9 mmHg in the control arm and about 5.6 mmHg in the intervention arm, about twice the decline. The arms start at similar levels (armintervention −1.32 mmHg, p = 0.22), as expected after randomization, so the difference in trends can be read as an effect of the program.

3. Run your own code: at the end of the script, type exp(coef(gee_a)[["visit"]] + coef(gee_a)[["armintervention:visit"]]) and exp(6 * coef(gee_a)[["armintervention:visit"]]) and run them. Interpret the odds ratio for armintervention:visit with its confidence interval. Then use your two results to give the odds ratio for one month in the intervention arm and the interaction odds ratio for one six-month interval between visits.

Model answerThe interaction odds ratio is 1.093 per month (95% CI 1.041 to 1.147): the odds of adherence change by a factor of about 1.09 per month more in the intervention arm than in the control arm, and the interval excludes 1, so the two arms change at clearly different rates. In the control arm the odds of adherence fall slightly, by about 1% per month (odds ratio 0.988), while your first line prints the monthly odds ratio in the intervention arm, 1.079 (the control-arm odds ratio combined with the interaction), a rise of about 8% per month. Your second line prints the interaction odds ratio for one six-month interval between visits, 1.70 (1.702627): odds ratios multiply over time, so this is the monthly interaction odds ratio of 1.093 applied six times, not 6 × 1.093. These are population-averaged odds ratios that describe the trial population as a whole, and 200 people give GEE plenty of clusters; the program is associated with a steady rise in adherence.
Saved.

Your output from Activity 5.4 reproduces the estimates in Tables 5.9 and 5.10 and shows how the missed visits were examined. The key takeaways below draw the section together before its knowledge check.

Key Takeaways

  • 1.2 Repeated measures are clustered data in which the cluster is the person. When someone is measured several times, every measurement carries that person's genes, habits, circumstances and baseline health, so their visits are more alike than visits from different people, and each person becomes a cluster of visits. This clustering is usually strong: in the wellness trial the ICC for blood pressure was 0.56, so more than half of the variation was a stable difference between people, compared with 0.175 for patients in clinics. In a question, the giveaway is a design in which the same people are measured at several times; the cluster is then the person, identified by a variable such as id, and not the visit or the year. Even a weak correlation between a person's visits, such as the 0.07 for adherence, still breaks the independence assumption, so an ordinary glm() is not the right choice.
  • 7.1 Repeated-measures data can be laid out in two ways, and the models in this lesson need one of them. In wide format each person has a single row with a column for each visit (such as sbp_0, sbp_6, sbp_12 and sbp_18), while in long format each visit has its own row, so a person with four visits appears four times. Mixed models and GEE in R need long format, with a column for the person identifier (the cluster), a column for the time of the visit and a column for the outcome, plus any predictors such as the study arm. Look at the first rows with head() before fitting anything, and convert wide data with reshape() or tidyr::pivot_longer(). If a question asks why the data have one row per visit, explain that the model follows change over time and needs the person identifier and the time on every row.
  • 7.2 Missed visits are common in repeated-measures studies, and each analysis treats them differently. A complete-case analysis drops everyone with a missed visit (79 of the 200 people in the trial), which wastes information and can bias the results if the people who missed visits differ from those who attended every one. A mixed model uses every visit that was attended (704 of the 800 planned measurements, from all 200 people) and gives valid results if visits are missing at random (MAR): the chance of missing a visit depends only on information the model already uses, such as the arm or earlier measurements, and not on the unrecorded value itself. GEE needs the stronger condition that missed visits depend only on its predictors, here arm and visit. MAR cannot be proved from the data, because it concerns values that were never recorded; comparing people who missed visits with those who did not shows which measured characteristics to include in the model, so state MAR in your answer as an assumption rather than as a fact.
  • 7.3 In a trial, the main question is usually whether the groups change at different rates, and the arm-by-time interaction answers it. In lmer(sbp_mmhg ~ arm * visit + (1 | id)), the arm term is the difference between the arms at the start (−1.32 mmHg, p = 0.22, so the arms start at similar levels, as expected after randomization), the visit term is the change per month in the reference (control) arm (−0.160 mmHg), and the arm:visit interaction is the extra change per month in the intervention arm (−0.149 mmHg, p = 0.014). A common trap is to read the arm term as the effect of the program, when it only describes the start. To make the interaction easier to picture, read it over the whole follow-up, which R gives with 18 * fixef(lmm_t)[["armintervention:visit"]] (−2.69, an extra fall of about 2.7 mmHg over 18 months), and read predicted values at a given time from the model, as in Worked Example 5.4. Because the arms were randomized, the interaction can be read as an effect of the program; when groups were not randomized, report it as an association.
  • 7.4 For a repeated yes-or-no outcome, GEE with the person as the cluster (id = id) gives population-averaged odds ratios, which suit a question about how the program changes adherence across participants, and 200 people provide plenty of clusters. The interaction odds ratio, 1.093 per month (95% CI 1.041 to 1.147), multiplies the control arm's monthly odds ratio, so the intervention arm's monthly odds ratio, which R gives with exp(coef(gee_a)[["visit"]] + coef(gee_a)[["armintervention:visit"]]), is 1.08, a rise of about 8% per month, while the control arm barely changes. Odds ratios multiply over time rather than add, so over a six-month interval the interaction odds ratio, from exp(6 * coef(gee_a)[["armintervention:visit"]]), prints 1.70, which is why a monthly odds ratio cannot simply be multiplied by six. Before reporting either repeated-measures model, check that the data are in long format and, for GEE, sorted by person and visit; that time is coded correctly and a straight-line trend is reasonable; the residual plots of the mixed model; the pattern of missed visits; the Number of clusters line for GEE; and that the people are independent of one another.
Knowledge check: this section

1. A trial randomized 240 adults with prediabetes to a group lifestyle program or usual care and weighed them at months 0, 3, 6 and 9. The first rows of the data are shown below, and the analysis is lmer(weight_kg ~ arm * month + (1 | id)). Why are the data arranged this way, and why does the model include (1 | id)?

idarmmonthweight_kg
P001usual092.4
P001usual392.0
P001usual6NA
P001usual991.6
P002program084.1
P002program383.0
Repeated weighings of one person share that person's body size and habits, so the person is the cluster. Mixed models need long format, and (1 | id) gives each person a baseline, allowing for the correlation among visits.
lmer() and geeglm() cannot use wide data, so wide data are reshaped to long format first. A wide row would also hide, not remove, the correlation among one person's repeated weighings.
Long format does not make the weighings independent. The random intercept (1 | id) is what allows for the correlation among a person's visits by giving each person their own baseline; it is not a label.
The visit time marks when a weighing was taken; it is not the cluster. The weighings of one person are alike because they share that person, so the person, identified by id, is the cluster.
Correct answer: each person is a cluster of visits, the models need long format, and (1 | id) gives each person their own baseline. The information needed is the design (the same people weighed four times) and the layout shown (one row per person per visit, with id repeated). A person's weighings share that person's body size and habits, so they are strongly alike; lmer() and geeglm() need long format, with columns for the person identifier (the cluster), the time and the outcome, and the random intercept lets each person have their own level. The wide layout is tempting, but these models cannot use it, so wide data are reshaped to long format first; the row with NA (P001 at month 6) is simply a missed visit. (Section 4, 1.2 Repeated Measures as Clustered Data and 7.1 Long and Wide Format, Table 5.8; Objectives 1.2 and 7.1)

2. In the weight trial, the analyst obtained the variances of the model and its ICC in R as shown below. What does the ICC mean, and why is it so much larger than the ICC of 0.175 for patients in the lesson's clinics?

> vc <- as.data.frame(VarCorr(m_wt)) > vc[, c("grp", "vcov")] grp vcov 1 id 63.04117 2 Residual 6.98522 > vc$vcov[1] / sum(vc$vcov) # ICC [1] 0.9002487
This reads the within-person share as the ICC. The ICC is the between-person share, so a person's repeated weighings are highly alike, not hardly alike.
An ICC can take any value from 0 to 1. Repeated measurements of one person share far more than patients in a clinic do, so large ICCs are usual for repeated measures and do not signal an error.
The ICC is the share of variation that lies between people. Repeated measurements of one person share that person's characteristics, so they are far more alike than patients who only share a clinic.
The ICC is a share of the variation in weight, not a share of participants. It says that most variation is a stable difference between people, not that 90% kept the same weight.
Correct answer: about 90% of the variation is a stable difference between people. The information needed is the ICC that R printed (0.90, the share of the two variances that lies between people, in the id row) and what a person's repeated measurements share. Two weighings of the same person are correlated at about 0.90; repeated measurements of one person share far more than patients in a clinic do, so their ICC is usually much larger, as the ICC of 0.56 for blood pressure in the wellness trial showed. The last option misreads the ICC as a share of people rather than a share of the variation, the first describes the within-person share, and an ICC can take any value from 0 to 1, so 0.90 is not a sign of an error. (Section 4, 1.2 Repeated Measures as Clustered Data and Activity 5.4; Objectives 1.2 and 3.1)

3. The fixed effects of m_wt <- lmer(weight_kg ~ arm * month + (1 | id)) are shown below, with usual care as the reference arm and month counted from the start, followed by three lines that give the change over the 9 months. Which coefficient answers whether weight falls faster in the program arm, and what does it say?

TermEstimate (kg)SEp-value
(Intercept)90.100.92< 0.001
armprogram−0.401.300.76
month−0.050.030.10
armprogram:month−0.200.04< 0.001
> 9 * fixef(m_wt)[["month"]] # usual care [1] -0.4464 > 9 * (fixef(m_wt)[["month"]] + fixef(m_wt)[["armprogram:month"]]) # program arm [1] -2.2581 > 9 * fixef(m_wt)[["armprogram:month"]] # difference [1] -1.8117
armprogram is the difference between the arms at month 0, which in a randomized trial is a chance difference at the start. Whether the program changes the trend is answered by the interaction, not by this term.
The interaction armprogram:month is the extra change per month in the program arm, so it answers whether the slopes differ; the R lines turn it into changes over 9 months, and randomization supports reading it as an effect.
The right term is chosen but its unit is misread. Because month is counted in months, the interaction is an extra change per month, which builds up over the 9 months rather than applying once.
The month coefficient is the change per month in the reference arm, usual care, not in both arms. The program arm's slope also includes the interaction, which shows its trend differs.
Correct answer: the armprogram:month interaction, an extra fall of about 1.8 kg over 9 months. The information needed is what each term measures: armprogram is the difference between the arms at month 0, month is the change per month in the reference (usual-care) arm, and armprogram:month is the extra change per month in the program arm, so only the interaction says whether the two slopes differ. The three R lines turn these per-month values into changes over the 9 months: about −0.45 kg with usual care, about −2.26 kg in the program arm, and a difference of about −1.8 kg; because the arms were randomized, the difference in trends can be read as an effect of the program. Reading armprogram as the program effect is the common error, because it is only the (chance) difference at the start, and the interaction is a change per month, not over the whole trial. (Section 4, 7.3 A Mixed Model for Change Over Time, Table 5.9 and Worked Example 5.4; Objective 7.3)

4. In the weight trial, 104 of the 960 planned weighings were missed, and only 170 of the 240 adults attended all four visits; the mixed-model output reports Number of obs: 856, groups: id, 240. People whose weight had risen at one visit were more likely to miss the next. Which statement is correct?

A complete-case analysis would drop 70 adults, and those who missed visits tended to be gaining weight, so the remaining group is selected and results are likely biased. Full information on fewer people is not safer here.
Data not missing completely at random can still be analysed validly. When missing depends on recorded information the model uses, such as earlier weights, the data are missing at random and a mixed model using all visits is valid.
GEE needs a stronger condition: missed visits must depend only on its predictors, arm and month. Here they depend on earlier weights, so that condition fails while the mixed model's condition can still hold.
The mixed model uses every attended visit and is valid when missing depends only on observed information it includes, as when earlier weights predict missed visits; this cannot be proven, because it concerns values never recorded.
Correct answer: the mixed model, which uses all 856 weighings and is valid if visits are missing at random. The information needed is how many weighings the model used (856, from the Number of obs line), how many adults were complete (170 of 240, so 70 would be dropped) and what predicts a missed visit (the earlier weight, which is recorded). A complete-case analysis would discard 70 people who tended to be gaining weight and is likely to be biased, while the mixed model uses every attended visit and is valid when missing depends only on observed information it includes; this cannot be confirmed, because it concerns weights that were never recorded. GEE needs the stronger condition that missed visits depend only on its predictors (arm and month), which fails when they depend on earlier weights. (Section 4, 7.2 Missed Visits, including the callout “Missing at Random, in Plain Terms” and the condition GEE needs; Objective 7.2)

5. The trial also recorded at each visit whether the adult met a physical activity target (yes or no). GEE with the person as the cluster (gee_w) gives an odds ratio per month of 0.98 in the usual-care arm and an interaction odds ratio (armprogram:month) of 1.10 per month (95% CI 1.04 to 1.16). The analyst then ran the two lines below. Which statement reads them correctly?

> exp(coef(gee_w)[["month"]] + coef(gee_w)[["armprogram:month"]]) # program arm, per month [1] 1.077744 > exp(3 * coef(gee_w)[["armprogram:month"]]) # interaction, per 3 months [1] 1.33048
An interaction describes how the trends differ over time, not a difference at every visit. The second line expresses the difference in trends over one 3-month interval, which says nothing about month 0.
The first line combines the usual-care slope and the interaction to give the program arm's own monthly odds ratio, and the second expresses the interaction over a 3-month interval; odds ratios combine by multiplying over time.
An odds ratio is a ratio of odds, not a change in a percentage. The value 1.08 describes how the odds of meeting the target change per month in the program arm.
Odds ratios multiply over time rather than add, so the interaction over 3 months is the monthly odds ratio raised to the third power, not three times it. The second line does this correctly.
Correct answer: about 1.08 per month in the program arm, and an interaction odds ratio of about 1.33 for a 3-month interval. The information needed is what each line computes. The first adds the usual-care slope and the interaction on the log-odds scale and exponentiates, which is the same as multiplying the two odds ratios (0.98 × 1.10), so it is the program arm's own monthly odds ratio. The second gives the interaction odds ratio over 3 months; odds ratios multiply over time rather than add, so it is 1.10 raised to the third power, not 3 × 1.10. An interaction describes how the trends differ, not a difference at the start, and an odds ratio is not a change in a percentage. The checks for this model include long data sorted by person and visit, and enough clusters (240 people is plenty for GEE). (Section 4, 7.4 GEE for a Repeated Binary Outcome, and the Checks for Repeated Measures, Table 5.10; Objective 7.4)

6. A second trial randomly assigned 300 adults aged 70 and over to a weekly walking group or to usual activities and measured their walking speed (m/s) at months 0, 4, 8 and 12. Of the 1,200 planned visits, 150 were missed, and 228 adults attended all four. As in Activity 5.4, the analyst compared the first-visit speed of adults who missed at least one later visit with that of adults who attended every visit. What does this comparison show?

> first <- walk[walk$month == 0, ] # each adult's first visit > round(tapply(first$speed, all4[first$id], mean), 2) # FALSE: missed a visit FALSE TRUE 0.81 0.96
The comparison shows only which recorded characteristics are related to missing visits. Missing at random concerns values that were never recorded, so it remains an assumption that no comparison can prove.
Missed visits that depend on earlier, recorded speeds are the situation in which a mixed model using every attended visit is still valid. Slower adults missing more visits does not make the data unusable.
The comparison shows that those who missed visits differed at the start, so complete cases describe a fitter group; the mixed model is valid if missing depends only on recorded information, which the data cannot confirm.
Completeness does not protect against bias. Those dropped were slower at the start, so the 228 complete cases describe a fitter group than the one randomized, and their results could be biased.
Correct answer: the complete cases are the fitter adults, and the mixed model is valid under missing at random, which the comparison cannot prove. The information needed is what the comparison can and cannot show: it shows which recorded characteristics are related to missing a visit (here a slower first visit), but missing at random concerns the values that were never recorded, so it remains an assumption. A complete-case analysis drops everyone with a missed visit, so when those people differ, here by being slower, its results describe a fitter group than the one randomized. Missed visits that depend on earlier, recorded measurements are exactly the situation in which a mixed model that uses every attended visit is still valid, so the data are not unusable. (Section 4, 7.2 Missed Visits and the callout “Missing at Random, in Plain Terms”; Objective 7.2)
● Complete the knowledge check to continue.
Final Assessment

Lesson 5: Final Assessment

16 questions • 100% required to pass

Bringing It All Together

This lesson addressed the assumption of independence that ordinary linear and logistic regression models share. Section 1 showed that observations are dependent when they share a cluster, such as patients in the same clinic or visits by the same person. The intracluster correlation coefficient measured the clustering in the clinic data (0.175 for systolic blood pressure), the design effect of 6.45 showed that the 966 patients carried about as much information as 150 independent patients, and an ordinary regression that ignored the clinics gave a standard error for urban clinics of 0.74 mmHg, compared with 1.96 mmHg from a model that allowed for them.

Section 2 fitted a linear mixed model with a random intercept for each clinic using lmer(). Its fixed effects were read like the coefficients of a linear regression, its random effects gave a clinic variance and a residual variance whose ratio was the adjusted ICC of 0.24, and it was checked with residual plots, a Q-Q plot of the clinic effects, the number of clinics and a test for a singular fit. Section 3 turned to a binary outcome, specialist referral, and compared a logistic mixed model fitted with glmer(), which gives cluster-specific odds ratios, with GEE fitted with geeglm(), which gives population-averaged odds ratios with sandwich standard errors. Section 4 treated each person in a wellness trial as a cluster of visits, arranged the data in long format, and used a mixed model for blood pressure and GEE for adherence, each with an arm-by-visit interaction that answered the main question of the trial and each using every visit that was attended.

Across the lesson, the same steps were followed each time: the cluster variable was identified, the clustering was measured, a model that allows for it was chosen to suit the outcome type and the research question, and the model's assumptions were checked before the results were reported. The final assessment asks for these steps to be applied to new studies.

Key Takeaways from this lesson

  1. You can now recognise clustered and repeated-measures data from the way a study was designed, such as patients recruited through clinics or people measured at several visits, and name the variable that identifies the clusters, which every method in this lesson needs. The most useful reminder for the exam is that R will not tell you: nothing in the output of an ordinary lm() or glm() warns that the data are clustered, so always begin by asking how the data were collected.
  2. You can explain why ignoring clustering usually leaves the estimates similar but makes the standard errors, confidence intervals and p-values too small, so that the model is overconfident, and you can sort predictors into cluster-level and person-level ones. Remember that the problem is largest for cluster-level predictors such as an urban clinic or a school-wide program, whereas for person-level predictors allowing for clustering changes the standard error little and can even make it smaller.
  3. You can interpret the ICC that R reports, read as the share of the outcome's variation that lies between clusters or as the correlation between two people in the same cluster, together with the design effect, 1 + (average cluster size − 1) × ICC, and the effective sample size, n ÷ DEFF. Judge an ICC against the usual 0.01 to 0.1 for clinics, schools and neighbourhoods, but remember that even a small ICC matters when clusters are large. Because the design effect grows with cluster size, adding clusters gains more precision than adding people to each cluster.
  4. You can read a random-intercept model fitted with lmer(y ~ x + (1 | cluster)): its fixed effects are interpreted like ordinary regression coefficients for people in the same cluster, its cluster and residual variances give the adjusted ICC, and an estimated cluster shift says how far one cluster's baseline lies above or below the average. Compared with an ordinary regression, expect similar estimates but a larger standard error for a cluster-level predictor. When you interpret a fixed effect, always say which people are being compared, and give the units and the confidence interval.
  5. You can describe how a mixed model is checked: with residual plots, a Q-Q plot of the cluster effects, enough clusters (about 20 to 30 or more) and a test for a singular fit, where isSingular() should return FALSE. You can also explain that a random slope lets the effect of a predictor differ between clusters. If a random slope brings a singular-fit or convergence warning, the model is too complex for the data, so report the simpler random-intercept model and say why.
  6. You can interpret odds ratios from a GLMM (glmer()) and from GEE (geeglm()): for a binary outcome, a GLMM gives cluster-specific odds ratios and GEE gives population-averaged odds ratios, which are usually a little closer to 1. You can choose between them from the research question: the GLMM suits questions about individual clusters, such as a clinic manager's, and GEE suits questions about the population, such as a provincial planner's. Before relying on GEE, check that there are enough clusters (about 30 to 40 or more) and that the data were sorted by cluster.
  7. You can interpret a repeated-measures analysis in long format, with one row per visit and the person as the cluster, including the group-by-time interaction, which tests whether the groups change at different rates. You can also explain why a mixed model that uses every attended visit is valid when missed visits are missing at random, whereas a complete-case analysis can be biased and GEE needs missed visits to depend only on its predictors. For the exam, remember that an interaction describes one unit of time, so read the change over the whole follow-up from R (a line such as 18 * fixef(model)[["arm:visit"]], or exp(6 * coef(model)[["arm:visit"]]) for an odds ratio, because odds ratios multiply over time), and state missing at random as an assumption, since the data cannot prove it.

The final assessment covers all four sections. All 16 questions must be answered correctly (100%) to finish the lesson.

Final Knowledge Assessment

Final assessment: the 16 questions

1. A health unit runs three studies: (A) a mailed survey of 1,500 adults drawn at random from the provincial health-insurance registry; (B) a dental survey of about 30 children in each of 50 child-care centres; (C) a study that measures the height of each of 300 children once a year for three years. Which classification is correct?

In A each adult is sampled separately from a registry, so there is no household clustering. In B the children are the members, not the cluster, and in C a shared measurer does not make one child's repeated heights independent.
Each study is classified by how people are selected and measured: separate sampling in A, groups sharing a centre in B, and repeated measurement of each child in C, where the child is the cluster.
A and B are right, but the year only marks when a measurement was taken. A child's three heights share that child's growth, so the child is the cluster in a repeated-measures design.
Random selection does not guarantee independence. Children sampled in groups from centres share staff, food and routines, and a child measured three times contributes alike measurements, so only A is independent.
Correct answer: A is independent, B is clustered by centre, and C is repeated measures with the child as the cluster. The information needed is how each study selects and measures people: in A each adult is sampled separately and measured once; in B children are sampled in groups that share a centre's staff, food and routines; in C each child is measured three times. The cluster is the unit whose members tend to be alike, which is the centre in B and the child in C. The option naming the year as the cluster is tempting, but the year only marks when a measurement was taken; a child's three heights share that child's growth, so the child is the cluster. (Section 1, 1.1 Recognising Dependent Data and Table 5.1, and Section 4, 1.2 Repeated Measures as Clustered Data; Objectives 1.1 and 1.2)

2. A provincial study measured the BMI z-score of 1,550 preschool children in 50 child-care centres (31 per centre). A mixed model with no predictors, m0, was fitted, and the ICC was obtained in R as shown below. Which statement is correct?

> vc <- as.data.frame(VarCorr(m0)) > vc[, c("grp", "vcov")] grp vcov 1 centre 0.05021 2 Residual 0.95411 > icc <- vc$vcov[1] / sum(vc$vcov) > icc [1] 0.04999403
The ICC is the centre variance as a share of the total, which is both the between-centre share of variation and the correlation between two children in a centre; 0.05 is an ordinary amount for such settings.
This reverses the two shares. The ICC is the between-centre share, so about 5% of the variation lies between centres and about 95% lies among children within centres.
The ICC is a share of the variation, not of the children. All children belong to a centre, so the clustering applies to the whole sample.
A typical ICC still matters when clusters are large. With 31 children per centre the design effect is well above 1, so ignoring the clustering would make standard errors for centre-level questions too small.
Correct answer: an ICC of about 0.05, with its two readings. The information needed is the value R printed for icc, the centre variance as a share of the total of the two variances (Equation 5.1), which is both the share of the variation that lies between centres and the correlation between two children in the same centre. Values of 0.01 to 0.1 are typical for clinics, schools and similar settings, so this is an ordinary amount of clustering. Concluding that it is small enough to ignore is the tempting error: with 31 children per centre it gives a design effect of about 2.5 (Question 3), so ignoring it would still make standard errors too small. The ICC is a share of the variation, not of the children. (Section 1, 3.1.1 The Intracluster Correlation Coefficient, Equation 5.1 and Worked Example 5.1; Objective 3.1)

3. In the child-care study, the analyst continued in R with the ICC from Question 2 and the average of 31 children per centre, as shown below. What do the design effect and the effective sample size mean?

> deff <- 1 + (31 - 1) * icc # design effect > deff [1] 2.499821 > 1550 / deff # effective sample size [1] 620.0445
This reverses the direction. The design effect inflates the variance of a centre-level estimate, so clustering reduces information; it never multiplies it.
The effective sample size is not a recruitment target. It expresses how much information the 1,550 children carry for centre-level questions, compared with independent children.
The design effect is the factor by which clustering inflates the variance of an estimate about a cluster characteristic, and the effective sample size is the number of independent children carrying the same information.
The effective sample size is not a count of particular children who were sampled independently. All 1,550 children together carry the information of about 620 independent children for centre-level questions.
Correct answer: a design effect of about 2.5 and an effective sample size of about 620 for questions about centre characteristics. The information needed is what each printed number means: the design effect (Equation 5.2) is the factor by which clustering inflates the variance of an estimate about a cluster characteristic, and the effective sample size, n ÷ DEFF, is the number of independent children who would carry the same information. The qualifier matters, because the loss of information applies to questions about centre characteristics, and the effective sample size is not a count of particular children or a recruitment target; clustering never adds information. (Section 1, 3.1.2 The Design Effect and the Effective Sample Size and Equation 5.2; Objective 3.1)

4. A new child-care study needs the precision of 600 independent children and expects an ICC of 0.05. Plan A recruits 41 children from each centre and Plan B recruits 11 children from each centre. The analyst multiplied 600 by each plan's design effect in R, as shown below. What does the comparison show?

> deff_A <- 1 + (41 - 1) * 0.05; deff_B <- 1 + (11 - 1) * 0.05 > c(A = 600 * deff_A, B = 600 * deff_B) # children needed A B 1800 900 > c(A = 600 * deff_A / 41, B = 600 * deff_B / 11) # centres needed A B 43.90244 81.81818
The number of centres matters a great deal: extra children from a centre already in the study partly repeat what its other children show, so fewer, larger centres need many more children for the same precision.
The ICC is assumed to be 0.05 in both plans. Plan A needs more children because the design effect grows with the number of children per centre, not because the ICC changes.
Both plans target the precision of 600 independent children, but the number of children needed to reach it differs because their design effects differ. A gap of this size is far too large to be rounding.
The required sample is the simple-random-sample size multiplied by the design effect, which grows with children per centre; spreading the sample over more centres therefore reaches the same precision with far fewer children.
Correct answer: Plan B reaches the same precision with half as many children, spread over more centres. The information needed is how a design effect is used in planning: the sample size needed from a simple random sample (600) is multiplied by the design effect, which is larger for Plan A (3.0) than for Plan B (1.5) because the design effect grows with the number of children per centre, even though the ICC (0.05) is the same in both plans. Each extra child from a centre already in the study partly repeats what the other children of that centre say, so adding clusters gains more precision than adding children per cluster. The number of centres matters for precision, and the gap between 1,800 and 900 is far too large to be rounding. (Section 1, 3.1.2 The Design Effect and the Effective Sample Size; Objective 3.1)

5. In the child-care study, 25 of the 50 centres have a healthy-snack policy. A colleague reports from lm() that the policy is associated with a lower BMI z-score (p = 0.02) and says the output showed no sign of a problem. The results from lm() and from a mixed model with a random intercept for each centre are shown below. What should you tell the colleague?

Predictorlm(): estimate (SE), plmer(): estimate (SE), p
Snack policy (centre)−0.12 (0.050), p = 0.02−0.13 (0.080), p = 0.11
Daily sugary drink at home (child)0.26 (0.055), p < 0.0010.25 (0.052), p < 0.001
lm() output never warns about clustering, which is recognised from the design. The similar estimates hide a large change in standard error for the centre-level policy, whose lm() p-value is too small.
The policy takes one value per centre, so its information comes from 50 centres; lm() treats the children as independent and is overconfident for it, while the child-level sugary-drink result barely changes.
The right model is chosen for the wrong reason. The two estimates are nearly identical; the mixed model is preferred because its standard error and p-value for the policy honestly reflect the 50 centres.
Choosing a model for each predictor by its p-value is selective reporting. The mixed model is the appropriate model for the whole analysis because the children are clustered in centres.
Correct answer: report the mixed model, for the reasons given. The information needed is the level of each predictor and how the results change: for the centre-level policy the estimate barely moves, but the standard error rises from 0.050 to 0.080 (about √2.5 ≈ 1.6 times, in line with the design effect of Question 3) and the p-value from 0.02 to 0.11, while the child-level sugary-drink result hardly changes. Ignoring clustering overstates the information about a cluster-level predictor, so lm() is overconfident, and the absence of a warning means nothing, because lm() output cannot show clustering. The estimates are nearly the same in both models, so preferring the mixed model for its larger estimate gives the wrong reason. (Section 1, the callout “Clustering Cannot Be Seen in the Regression Output” in 1.1, and 2.1 to 2.3 with Table 5.2; Objectives 1.1, 2.1, 2.2 and 2.3)

6. In the mixed model of Question 5, the snack-policy coefficient is −0.13 (95% CI −0.29 to 0.03), adjusted for age, sex and daily sugary drinks. The centres chose whether to adopt the policy. Which interpretation is correct?

This causal reading ignores two points: the CI includes 0, so the data are compatible with no difference, and the centres chose their policies, so other differences between centres could explain the result.
A negative estimate is not the same as a significant one. The 95% CI includes 0, so the data are compatible with no difference.
A fixed coefficient is a difference in average z-score, not a percentage of variation explained. Shares of variation come from the variance components.
A fixed effect is read as an adjusted difference in average outcome; with an interval including 0 the data cannot show a clear difference, and self-selected policies make the comparison observational.
Correct answer: about 0.13 lower on average, with a confidence interval that includes 0 and an observational design. The information needed is the estimate, its 95% CI (−0.29 to 0.03, which spans 0) and how the policy was assigned. A fixed effect is read like an ordinary regression coefficient, as a difference in the average outcome for children with the same values of the other predictors; because the interval includes 0, the data are compatible with no difference, and because centres chose the policy, other differences between centres could explain part of any difference. The causal reading in the first option ignores both the interval and the design. (Section 2, 4.2 Interpreting the Fixed Effects and Cluster Shifts and Table 5.4; Objective 4.2)

7. From the same mixed model (sugary-drink coefficient 0.25), the model's predicted BMI z-scores for two 4-year-old boys in centres without the policy, one without and one with a daily sugary drink, are shown below for an average centre and for centre C14, whose estimated shift is −0.30. What do the predictions show?

BoyAverage centreCentre C14
No daily sugary drink0.400.10
Daily sugary drink0.650.35
A centre shift is shared by everyone in the centre, so it moves every prediction for C14 by the same amount and cancels out of comparisons within C14, leaving the sugary-drink difference unchanged.
A random intercept shifts the baseline only. It cannot change the sugary-drink difference; that would require a random slope for the sugary-drink term.
The shift is the part of C14's difference from an average centre that the predictors do not explain. The policy is a measured predictor with its own coefficient, and both boys here are in centres without it.
The model's outcome is a BMI z-score, so the shift is a difference in z-score, not a percentage or a probability of being overweight.
Correct answer: the shift lowers both predictions by the same amount, and the sugary-drink difference stays 0.25. The information needed is the four predictions and what a cluster shift is: C14's shift of −0.30 is shared by everyone in the centre, so it moves every prediction for C14 down by 0.30 (0.40 to 0.10 and 0.65 to 0.35), and the difference between the two boys is 0.25 in both columns. The shift describes a difference between C14 and an average centre that the predictors in the model do not explain, not the effect of a measured variable such as the policy, which has its own coefficient; it is a difference in z-score, not a percentage. (Section 2, 4.2 Interpreting the Fixed Effects and Cluster Shifts and Worked Example 5.2; Objective 4.2)

8. The mixed model for the 50 centres passed its checks: the residual plots looked acceptable, the Q-Q plot of the centre effects followed the line, and isSingular() returned FALSE. A colleague plans to fit the same model to a pilot study of 10 centres. What is the main concern?

The number of children does not fix the problem. The centre variance and any centre-level predictor draw their information from the centres, and 10 centres is well below the usual minimum.
Residual plots can be drawn whatever the number of clusters. The concern is that the centre variance is estimated poorly with 10 centres and the Q-Q plot of 10 centre effects is hard to judge.
The cluster variance is estimated from the clusters themselves, so 10 centres estimate it poorly, make a singular fit more likely and leave too few centre effects for a useful Q-Q check.
This reverses the requirement. GEE needs more clusters than a mixed model, about 30 to 40 or more, so switching to it with 10 centres would make things worse.
Correct answer: 10 centres are too few for the centre variance to be estimated well. The information needed is the rule for the number of clusters: the cluster variance is estimated from the clusters, so a model with about 10 clusters or fewer estimates it poorly, and about 20 to 30 is a common minimum. The number of children does not fix this, because a centre-level variance (and a centre-level predictor) draws its information from the centres; with few clusters a singular fit is also more likely, and the Q-Q check of the cluster effects becomes rough. Switching to GEE would make things worse, because GEE needs more clusters (about 30 to 40 or more), not fewer. (Section 2, 5.1 Checking a Linear Mixed Model, Table 5.5 and the slide “Enough clusters, and no singular fit”; Objective 5.1)

9. To see whether the sugary-drink difference varies between centres, the analyst fits lmer(zbmi ~ age + girl + sugary + policy + (sugary | centre)). R warns that the model failed to converge, and the sugary-drink estimate is 0.25 in both this model and the random-intercept model. Which statement is correct?

A model that failed to converge gives no reliable information about the slopes. The warning means the random part is more than the data can support, not that the sugary-drink difference is identical everywhere.
A random slope lets an association vary between clusters, but a convergence warning shows that R could not settle on reliable estimates for that extra term, so the random-intercept model is reported and the attempt noted.
A model that failed to converge is not the one reported, whether or not the fixed effects change. The matching estimates only show that the conclusion does not depend on the random slope.
A convergence warning concerns the random part of the model, not the distribution of the outcome. It is not a reason to switch to GEE.
Correct answer: report the random-intercept model and note that the random slope could not be estimated. The information needed is what (sugary | centre) adds (each centre gets its own sugary-drink slope as well as its own baseline) and what a convergence warning means: R could not settle on reliable estimates, which, like a singular fit, signals that the random part of the model is more than the data can support. Because the sugary-drink estimate is the same in both models, the conclusion does not depend on the random slope, but the model that failed to converge is not the one reported. Reading the warning as evidence that the slope is the same everywhere is wrong, because a failed fit says nothing reliable about the slopes. (Section 2, 5.2 Random Slopes and Model Warnings; Objective 5.2)

10. Overweight (yes or no) was modelled with a GLMM (glmer(), a random intercept for each centre) and with GEE (geeglm(), 50 centres). The odds ratio for a daily sugary drink is 1.60 (95% CI 1.30 to 1.97) in the GLMM and 1.52 (1.26 to 1.83) in GEE. A centre director asks how a daily sugary drink relates to overweight for children in her centre; a provincial planner asks how overweight differs across the province between children with and without a daily sugary drink. Which statement is correct?

This reverses the two estimates. The GLMM, with a baseline for each centre, compares children within a centre, which suits the director; GEE describes the whole population, which suits the planner.
GEE does allow for clustering, through sandwich standard errors. Its odds ratio is closer to 1 because averaging over centres with different baselines pulls a population-averaged comparison toward 1.
Odds ratios are not ratios of probabilities, and the two answer different questions, within-centre and population-wide. The choice depends on who is asking.
The GLMM odds ratio is cluster-specific and suits a question about one centre, while the GEE odds ratio is population-averaged and suits a province-wide question; it is usually a little closer to 1, and 50 centres suffice.
Correct answer: the GLMM (cluster-specific) odds ratio for the director and the GEE (population-averaged) odds ratio for the planner. The information needed is the two odds ratios, what each model compares and who is asking: the GLMM gives each centre its own baseline, so its odds ratio compares children who share a centre, while GEE fits one model for the population with standard errors that allow for clustering, so its odds ratio compares groups across all centres. For a binary outcome, averaging over centres with different baselines pulls the population-averaged odds ratio toward 1 (1.52 against 1.60), and GEE needs about 30 to 40 clusters or more, which 50 centres provide. Both models allow for clustering, so the claim that GEE ignores it is wrong, and odds ratios are not ratios of probabilities. (Section 3, 6.3 Clinic-Specific and Population-Averaged Odds Ratios with Worked Example 5.3, and 6.4 Choosing Between a GLMM and GEE; Objectives 6.3 and 6.4)

11. For the snack policy, ordinary logistic regression (glm()) gives an odds ratio for overweight of 0.84 (95% CI 0.71 to 0.99), and GEE with the centre as the cluster gives 0.85 (95% CI 0.64 to 1.13). Why is the GEE interval wider, and which result should be reported?

For a centre-level predictor, glm() overstates the information by treating children as independent, while GEE's sandwich standard error reflects variation between the 50 centres. The GEE interval includes 1, so no clear association.
Both models use the same children. GEE's interval is wider because it allows for clustering; choosing glm() because its interval excludes 1 rewards an interval that is too narrow.
Population-averaged odds ratios do not always have wider intervals. The GEE interval is wider here because it allows for clustering of a centre-level predictor, and averaging a valid and an invalid result is not a remedy.
The two results are not equally valid: glm() ignores the clustering, so its interval is too narrow. Choosing whichever interval excludes 1 is the tempting error.
Correct answer: report GEE (or a GLMM), which shows no clear association. The information needed is that the policy is a centre-level predictor and that the two odds ratios are almost the same (0.84 and 0.85) while the interval widens from 0.71 to 0.99 to 0.64 to 1.13. glm() assumes that the 1,550 children are independent, so for a predictor that takes one value per centre it overstates the information and gives an interval that is too narrow; GEE's sandwich standard error comes from the variation between the 50 centres. Choosing a result because its interval excludes 1 is the tempting error: the glm() interval excludes 1 only because it ignores the clustering. (Section 3, 6.2 Odds Ratios and Standard Errors from the Three Models and Table 5.6; Objectives 2.2 and 6.2)

12. In a separate community study, 360 children were measured once a year for four years. The data file has one row per child with the columns id, sport, z_0, z_1, z_2 and z_3 (the BMI z-score in years 0 to 3), where sport = 1 if the child's parents enrolled them in a free after-school sports program. What must be done before fitting lmer(zbmi ~ sport * year + (1 | id)), and what is the cluster?

lmer() needs long format, with one row per measurement. A wide row per child does not make the child an independent observation suitable for this model.
Averaging the four measurements would throw away the change over time that the study is about, and the model with sport * year needs each yearly measurement.
lmer() needs long format, one row per measurement with columns for the identifier, time and outcome. Each child's four measurements share that child's characteristics, so the child is the cluster.
Reshaping to long format is right, but the year is the time of a measurement, not a cluster. The repeated measurements of one child are alike, so the child is the cluster.
Correct answer: reshape to long format; the child is the cluster. The information needed is the current layout (wide, one row per child with a column for each year) and what the model needs: lmer() and geeglm() need long format, with one row per measurement and columns for the person identifier, the time and the outcome, which reshape() or tidyr::pivot_longer() can produce. Each child's four measurements share that child's characteristics, so the child is the cluster and (1 | id) gives each child their own baseline. Averaging the four columns would throw away the change over time that the study is about, and the year is the time of a measurement, not a cluster. (Section 4, 7.1 Long and Wide Format; Objectives 1.2 and 7.1)

13. The model m_z <- lmer(zbmi ~ sport * year + (1 | id)) gives a coefficient of 0.03 per year for year (p = 0.04) and −0.04 per year for sport:year (95% CI −0.07 to −0.01). Parents chose whether to enrol their child in the sports program. The analyst then ran the lines below. Which interpretation is correct?

> fixef(m_z)[["year"]] + fixef(m_z)[["sport:year"]] # yearly change, program children [1] -0.0098 > 3 * fixef(m_z)[["sport:year"]] # difference in change over 3 years [1] -0.1206
Families chose whether to enrol their child, so the groups were not randomized and other differences, such as diet or income, could explain part of the difference in trends. The result is an association.
The reference-group slope gives the trend without the program, the interaction gives the extra change per year with it, and the R lines express these over time; because families chose the program, it is an association.
The sport term is the difference between the groups at year 0. Only the interaction sport:year says whether the yearly trends differ.
The interaction is a difference per year, not over the whole study. Over three years the difference in trends builds up, as the second R line shows.
Correct answer: the trends differ by about 0.04 per year, or about 0.12 over three years, and this is an association because the groups were not randomized. The information needed is the reference-group slope (0.03 per year), the interaction (−0.04 per year, the extra change per year in the sports group), the two R lines and how the groups were formed. The first line gives the program children's own yearly change (about −0.01), and the second turns the per-year interaction into a difference over three years (about −0.12); the interval for the interaction excludes 0. The causal reading is tempting, but when groups are not randomized, families who chose the program may differ in diet, income or other habits, so the result is reported as an association; the last option forgets that the interaction is per year. (Section 4, 7.3 A Mixed Model for Change Over Time, Table 5.9, Worked Example 5.4 and the paragraph that follows it; Objective 7.3)

14. In the community study, 150 of the 1,440 planned measurements were missed, and children who missed a measurement tended to have had higher z-scores at their previous one. The mixed model used all 1,290 measurements that were taken. Which statement is correct?

This describes missing completely at random, not missing at random, and it overstates the comparison: comparing those who missed with those who did not shows which recorded characteristics predict missing but cannot confirm MAR.
A complete-case analysis would keep only children measured every year, who tended to have lower z-scores, and could be biased. Missing that depends on earlier recorded z-scores is the situation in which the mixed model is valid.
Dependence on the unrecorded value itself is missing not at random (MNAR), not MAR. Here missing depends on the previous, recorded z-score, which fits MAR.
MAR means missing depends only on recorded information the model uses; earlier z-scores fit this, so the mixed model using every measurement is valid if MAR holds, although MAR concerns unrecorded values and cannot be proven.
Correct answer: missing at random, which the description fits but the data cannot prove. The information needed is what predicts a missed measurement (the previous z-score, which was recorded) and the three missing-data mechanisms: MCAR (missingness unrelated to any value), MAR (it depends only on observed values) and MNAR (it depends on the missing value itself). A mixed model that uses all 1,290 measurements is valid under MAR, whereas a complete-case analysis would keep only the children who were measured every year, who tended to have lower z-scores, and could be biased. Comparing children who missed measurements with those who did not shows which recorded characteristics predict missing, but it cannot confirm MAR, because MAR concerns values that were never recorded; the first option describes MCAR and overstates what the comparison can show. (Section 4, 7.2 Missed Visits and its callouts “Recall: Missing-Data Mechanisms” and “Missing at Random, in Plain Terms”; Objective 7.2)

15. In the community study, whether each child met the daily activity guideline (yes or no) at each yearly measurement was analysed with GEE (gee_g), with the child as the cluster. The odds ratio per year is 0.90 for children not in the sports program, and the interaction odds ratio (sport:year) is 1.25 per year (95% CI 1.08 to 1.45). The analyst then ran the two lines below. Which statement is correct?

> exp(coef(gee_g)[["year"]] + coef(gee_g)[["sport:year"]]) # program children, per year [1] 1.125357 > exp(3 * coef(gee_g)[["sport:year"]]) # interaction, over 3 years [1] 1.951113
The first line combines the reference slope and the interaction to give the program group's own yearly odds ratio, and the second expresses the interaction over three years; odds ratios multiply over time, and 360 clusters are plenty.
Odds ratios multiply over time rather than add, so the interaction over three years is the yearly odds ratio raised to the third power, not three times it. The second line is correct.
An odds ratio is a ratio of odds, not a proportion of children. The first line gives the program group's yearly odds ratio, not the share who met the guideline.
An interaction describes how the yearly trends differ, not a difference at every measurement. It says nothing about the groups at year 0.
Correct answer: about 1.13 per year in the program group, and an interaction odds ratio of about 1.95 over three years. The information needed is the per-year odds ratio in the reference group (0.90), the interaction odds ratio (1.25), the factor by which the program group's yearly odds ratio differs from the reference group's, and what each R line computes. The first line combines the two on the log-odds scale and exponentiates, which multiplies the odds ratios (0.90 × 1.25), so it is the program group's own yearly odds ratio; the second gives the interaction over three years, and because odds ratios multiply over time it is 1.25 raised to the third power, not 3 × 1.25. An odds ratio is not a proportion of children, and the interaction describes how the trends differ, not a difference at the start; these are population-averaged odds ratios, and, as in Question 13, the comparison is observational. (Section 4, 7.4 GEE for a Repeated Binary Outcome, and the Checks for Repeated Measures, Table 5.10; Objective 7.4)

16. A city surveyed 1,800 adults in 45 neighbourhoods (40 per neighbourhood) about food security. Community groups run a food hub, a characteristic of the neighbourhood, in 20 of them, and a mixed model with no predictors gives an ICC of 0.05 for the food-security score. The standard error of the hub coefficient is 0.54 in lm() and 0.92 in lmer() with a random intercept for each neighbourhood. The analyst ran the lines below. What do they show?

> deff <- 1 + (40 - 1) * 0.05 # design effect > deff [1] 2.95 > sqrt(deff) # expected inflation of the standard error [1] 1.717556 > 0.92 / 0.54 # SE of the hub in lmer() divided by SE in lm() [1] 1.703704
The mixed model is not less efficient; lm() is overconfident. For a neighbourhood-level predictor, lm() treats the adults as independent, so its smaller standard error comes from a false assumption.
A typical ICC still matters when clusters are large. With 40 adults per neighbourhood the design effect is well above 1, so the standard error for a neighbourhood characteristic must allow for it.
A design effect inflates the variance of a cluster-level estimate, so the standard error grows by about its square root; the mixed model's standard error matches this, showing that lm() is too confident about the hub.
A ratio of standard errors describes how uncertainty changes, not the size of the association. The hub estimate itself is a separate quantity from its standard error.
Correct answer: the design effect predicts a standard error about 1.7 times as large, and that is what the mixed model shows. The information needed is what a design effect means and how it relates to a standard error: it is the factor by which clustering inflates the variance of an estimate about a cluster characteristic, so the standard error grows by about its square root, as the design effect of 6.45 and the factor of about 2.5 did for the clinics. The hub takes one value per neighbourhood, so the 1,800 adults supply only 45 independent values of it, and lm(), which treats them as independent, is overconfident. A small ICC still matters when clusters are large (40 adults each), and a ratio of standard errors says nothing about the size of the estimate. (Section 1, 2.2 Cluster-Level and Person-Level Predictors and 3.1.2 The Design Effect and the Effective Sample Size; Objectives 2.2 and 3.1)

✦ Before submitting: pass every section knowledge check (100%).

🏆 Congratulations!

Linear and logistic regression models can now be applied when people are grouped in clinics, schools or neighbourhoods, or are measured more than once.

You have successfully completed this lesson: Modelling Dependent Data.

Your responses have been downloaded automatically.