HSCI 410 · Lesson 6

Exploratory Data Analysis and Visualization

Exploratory Data Analysis For Epidemiology

Learning objectives for this lesson:

  1. Explain the purpose of exploratory data analysis, how it differs from confirmatory analysis, and why summary statistics alone can mislead, using Anscombe's quartet as the illustration.
  2. Choose an appropriate display for one variable, for two variables and for many variables from the question and the types of the variables involved, and decide whether to show groups by colour or in separate panels (facets).
  3. Identify common poor displays (a truncated axis, bars of means without spread, a pie chart with many slices, overplotting, a dual axis, and colours that rely on telling red from green), describe what each conceals, and propose a better display.
  4. Read histograms and boxplots for shape, skew, outliers, ceilings, heaping, top-coding, impossible values and missing-value codes, and describe how to check, decide and report each problem and what it suggests for the later analysis.
  5. Explain in plain words what each part of the base R or ggplot2 code for a figure does (variables mapped to axes, colour or fill; boxes, jittered and see-through points, and smoothed lines with their confidence bands; facets, palettes, themes and saving with code), and describe what the resulting figure shows, including when a panel holds too few people to interpret.
  6. Interpret a correlation matrix, a correlation heatmap and a scatterplot with marginal histograms: describe a correlation by its direction and strength, explain when Spearman's correlation is preferred, and explain why a correlation cannot show a cause, a curved relationship or the shape of the data.
  7. Critique a combined multi-panel figure against the conventions of a publication figure, and write a caption that lets the figure stand on its own while leaving interpretation to the results text.

This course was developed by Dr. Kiffer G. Card based on Dohoo, I. R., Martin, S. W., & Stryhn, H. (2012). Methods in Epidemiologic Research. VER Inc.

Lesson 6 · HSCI 410

Exploratory Data Analysis and Visualization

Looking at the data before modelling them, and turning what you see into a figure that others can read.

Exploratory Data Analysis For Epidemiology
The running case

A visual briefing for a loneliness working group

The request

The working group wants a one-page visual briefing on social connection before any modelling is commissioned.

The data

The briefing uses the 2021 wave of the Canadian Social Connection Survey (CSCS), a national online survey whose respondents were not sampled at random.

Each section adds plots to the briefing, and Section 4 assembles them into a single figure.

Lesson map

Four sections

SectionWhat it covers
1. Look Before You ModelExploratory analysis, Anscombe's quartet, choosing a display, and poor displays
2. Quick Looks in Base Rhist(), barplot(), boxplot(), plot(), pairs() and saving plots
3. The Grammar of Graphics with ggplot2Aesthetics, geoms, facets, colour, themes and ggsave()
4. Many Variables and Combined FiguresCorrelation heatmaps, marginal histograms, multi-panel figures and captions
Working in R

Every line gives you something to look at

  • Each activity gives the complete code, with a comment on every line.
  • The console output shown on the page is what R printed when the code was run.
  • The questions ask you to read and interpret the plots and the output.
  • A narrated walkthrough, Data Visualization in Base R and ggplot2, follows the same code.
Reference

Glossary: Key Terms, People & Concepts

📚 Reference page, available throughout the lesson

This glossary collects the key concepts, methods and people in this lesson. It can be used as a reference while working through the material or as a review before assessments. Typing in the search box filters the entries.

Key Concepts & Ideas
Exploratory Data Analysis Exploratory data analysis uses plots and simple summaries to learn the shape, quality and relationships of the data before any formal model is fitted. John Tukey promoted the approach from about 1970 and set it out in his book Exploratory Data Analysis (1977).
Confirmatory Analysis Confirmatory analysis tests a question that was set before the data were examined and reports estimates, confidence intervals and p-values. It usually follows exploration.
Anscombe's Quartet Anscombe's quartet is a set of four small datasets with nearly identical means, standard deviations, correlations and fitted lines but very different patterns when plotted. It is supplied with R as anscombe.
Shape of a Distribution The shape of a distribution describes whether values are symmetric or skewed, how many peaks they have, and whether they pile up at the lowest or highest possible value.
Outlier An outlier is a value far from most of the others. It may be a recording error or a genuine extreme, and it is checked before any decision is made about it.
Impossible or Implausible Value An impossible value lies outside the possible range of a variable, and an implausible value is possible but very unlikely, such as more than 112 hours of paid work a week.
Heaping (Digit Preference) Heaping occurs when answers pile up on round or standard numbers, such as 40 hours of work a week. It usually reflects rounding, and it reduces the information carried by small differences.
Ceiling and Floor A ceiling or floor is a pile-up of answers at the highest or lowest value a scale allows. On the three-item loneliness scale, 270 people in the briefing file scored the maximum of 9.
Top-Coding Top-coding records every value above a limit as the limit itself. In the CSCS data, household totals above 20 were recorded as 20, which produced a spike at that value.
Overplotting Overplotting occurs when many points are drawn on top of one another, so the plot cannot show where most of the data lie. Jitter and transparency reduce it.
Small Multiples Small multiples are a series of small panels that repeat the same display for different groups on shared axes, a term associated with Edward Tufte. In ggplot2 they are drawn with facets.
Chartjunk Chartjunk is Tufte's term for decoration in a graphic that carries no information, such as three-dimensional effects, heavy shading or ornaments.
Association and Causation An association means that two variables tend to vary together. It may arise because one causes the other, because of reverse causation, or because a common cause (a confounder) affects both.
Colour-Vision Deficiency Colour-vision deficiency is a reduced ability to tell some colours apart, most often red from green. It affects about 8% of men and 0.5% of women of northern European ancestry.
Figure Caption A figure caption states what a figure shows, the data source, the number of people included, and what each panel and mark means, so that the figure can be understood on its own.
Methods & Statistical Concepts
Histogram A histogram divides the range of a numeric variable into bins and draws a touching bar for the number of values in each. It is drawn with hist() in base R and geom_histogram() in ggplot2.
Bar Chart A bar chart draws one separate bar per category of a categorical variable. In base R it is drawn with barplot(table(x)), and in ggplot2 with geom_bar() or geom_col().
Boxplot A boxplot summarizes a numeric variable with its median, quartiles and whiskers, and draws values beyond the whiskers as separate points. Side-by-side boxplots compare groups.
Interquartile Range and Whiskers The interquartile range is the distance between the lower and upper quartiles, the length of the box. Whiskers reach the most extreme values within 1.5 times the interquartile range of the box.
Scatterplot A scatterplot places one point per observation for two numeric variables, showing the direction, form and strength of their relationship.
Scatterplot Matrix A scatterplot matrix draws a small scatterplot for every pair of several variables in a grid. In base R it is drawn with pairs().
Jitter Jitter adds a small random shift to each plotted value so that points sharing a value spread out. It changes only the picture and leaves the data unchanged.
Transparency (Alpha) Transparency draws points so that they can be seen through, so that areas where many points overlap look darker. It is set with alpha in ggplot2 or rgb(..., alpha) in base R.
Grammar of Graphics The grammar of graphics describes every statistical graphic as a combination of data, aesthetic mappings, geometric objects, scales, facets and a theme. ggplot2 implements a layered version of it.
Aesthetic Mapping An aesthetic mapping, written inside aes(), links a variable to a visual property such as horizontal position, colour or fill. A value given outside aes() is set for every mark.
Geom A geom is the type of mark that represents data in ggplot2, such as geom_histogram(), geom_boxplot(), geom_point() or geom_tile(). Each geom is a layer.
Facets Facets split a ggplot2 plot into panels by one or two categorical variables. facet_wrap() uses one variable, and facet_grid() places one variable in rows and another in columns.
Theme The theme controls the parts of a ggplot2 plot that do not depend on the data, such as fonts, text size, grid lines and background, for example theme_minimal().
Loess Smoother A loess smoother is a curve that follows the average of y across x by fitting many small weighted regressions, so it can bend. It is drawn with geom_smooth(method = "loess").
Pearson Correlation Coefficient The Pearson correlation coefficient, r, measures how closely two numeric variables follow a straight line, from −1 to +1. It is computed with cor().
Spearman's Rank Correlation Spearman's rank correlation is the Pearson correlation of the ranks of two variables. It suits ordinal or skewed variables and is less affected by extreme values.
Correlation Matrix A correlation matrix holds the correlation of every pair of several variables, with 1 on the diagonal. It is symmetric, so k variables give k(k − 1)/2 distinct correlations.
Correlation Heatmap A correlation heatmap draws a correlation matrix as coloured squares, usually with a diverging colour scale that is white at zero. It shows association and cannot identify causes.
Marginal Histogram A marginal histogram is drawn along the edge of a scatterplot to show the distribution of one of its variables. ggExtra::ggMarginal() adds them to a ggplot2 scatterplot.
patchwork patchwork is an R package that combines ggplot2 plots into one figure: + or | places plots side by side and / stacks them.
ggsave() and Resolution ggsave() writes a ggplot2 plot to a file of a chosen width and height. Resolution, in dots per inch (dpi), is usually 300 for print.
Okabe-Ito Palette The Okabe-Ito palette is a set of eight colours chosen so that people with common forms of colour-vision deficiency can tell them apart.
Bracket Recoding and Complete Cases Bracket recoding changes the rows of a variable that meet a condition, as in x[x == 20] <- NA. na.omit() keeps only rows with no missing values (a complete-case file).
Key People
John W. Tukey (1915–2000) John Tukey was an American statistician at Princeton University and Bell Laboratories. He introduced the boxplot, first called the schematic plot, in about 1970, and his book Exploratory Data Analysis (1977) set out the approach that gives the field its name.
Francis J. Anscombe (1918–2001) Francis Anscombe was a British-born statistician at Yale University. His 1973 paper in The American Statistician presented the quartet that bears his name.
Leland Wilkinson (1944–2021) Leland Wilkinson was a statistician and computer scientist who wrote The Grammar of Graphics (1999; second edition 2005) and developed the SYSTAT statistical software.
Hadley Wickham Hadley Wickham is a statistician and software developer who created ggplot2 and described the layered grammar of graphics (2010). He is chief scientist at Posit, formerly RStudio.
William S. Cleveland William Cleveland is a statistician who studied graphical perception with Robert McGill (1984) and developed the loess method of local regression smoothing.
Edward R. Tufte Edward Tufte is a statistician and political scientist and professor emeritus at Yale University. The Visual Display of Quantitative Information (1983) introduced the terms data-ink and chartjunk.
Masataka Okabe and Kei Ito Masataka Okabe and Kei Ito are Japanese scientists who proposed a colour palette for Color Universal Design, first published in 2002 and revised in 2008, that remains distinguishable for people with colour-vision deficiency.
No matching entries. Try a different search term.
Section 1 of 4

Look Before You Model

⏱ Estimated time: 45 minutes
Lesson 6 · Section 1

Look Before You Model

Plots show the shape of the data, and the shape decides which summaries and models make sense.

Exploratory data analysis

Two kinds of analysis

Exploratory

The analyst asks what the data look like.

Plots and simple summaries show shapes, unusual values and relationships, and they raise new questions.

Confirmatory

The analyst tests a question that was set in advance.

Models give estimates, confidence intervals and p-values for that question.

Tukey (1977) argued that exploration should come first.

The running case

The working group's three questions

  • The group asks how lonely people in the survey are.
  • The group asks whether loneliness differs by age and gender.
  • The group asks how loneliness goes together with social support and mental health.
4,045
respondents in the CSCS 2021 wave
3,083
respondents with complete data on the briefing variables
Anscombe's quartet

Four datasets with the same summary statistics

StatisticSet 1Set 2Set 3Set 4
Mean of x and of y9, 7.59, 7.59, 7.59, 7.5
SD of x and of y3.32, 2.033.32, 2.033.32, 2.033.32, 2.03
Correlation0.8160.8160.8160.817
Fitted liney = 3.00 + 0.500 x in every set
Anscombe's quartet

The plots tell four different stories

Four scatterplots of Anscombe's quartet, each with the same fitted line: set 1 a linear cloud, set 2 a curve, set 3 a line with one outlier, set 4 a vertical column of points with one far point.
The same fitted line (red) suits only set 1. Data: Anscombe (1973), supplied with R as anscombe.
What to look for

Features that plots reveal

One variable

The analyst checks the shape (symmetry, skew and peaks), unusual and impossible values, heaping on round or extreme values, and missing values.

Two or more variables

The analyst checks the direction of a relationship, its form (straight or curved), its strength, and whether it differs between groups.

Choosing a display

Match the display to the question and the variable types

Recall: Lesson 2, Section 3

A histogram shows one numeric variable, a bar chart the counts of a categorical variable, boxplots a numeric variable across groups, and a scatterplot two numeric variables.

New: many variables

A scatterplot matrix or a correlation heatmap gives an overview, and facets repeat a display for each group.

Check: loneliness across four age groups calls for boxplots side by side, with the points shown.

Poor displays

Four displays that conceal the data

Four poor displays: a bar chart with a truncated axis, a bar chart of means without spread, a pie chart with 14 slices, and an overplotted scatterplot.
A truncated axis, means without spread, too many slices and overplotting (CSCS 2021).
Carry forward

What to take into the next section

  • Identical summary statistics can describe very different data, so the analyst plots first.
  • The display is chosen from the question and the types of the variables involved.
  • Some displays conceal the data, and each has a better alternative.

Overview

Regression models for outcomes of many kinds rest on assumptions about the shape of the data: a straight-line relationship, residuals with an even spread, counts that are not more variable than the model allows, or observations that are independent. This lesson turns to the step that should come before any model is fitted, which is looking carefully at the data. Exploratory data analysis uses plots and simple summaries to find the shape of each variable, unusual or impossible values, and the relationships between variables. It tells the analyst whether the planned model suits the data, and it often suggests questions that the model should answer. This section explains what exploration is and how it differs from confirmatory analysis, uses Anscombe's quartet to show why summary statistics alone can mislead and reproduces the quartet in R, sets out how the number and the type of the variables in a question point to a display and when groups are better shown by colour or in separate panels, and ends with a gallery of common displays that conceal more than they reveal.

Learning Objectives

  • 1.1 Explain the purpose of exploratory data analysis and how it differs from confirmatory analysis.
  • 1.2 Use Anscombe's quartet to explain why summary statistics alone can mislead.
  • 1.3 Compute summary statistics and draw a set of four plots in R for Anscombe's quartet, and say what the plots show that the statistics do not.
  • 2.1 Choose an appropriate display for one variable, for two variables and for many variables, given the variable types involved.
  • 2.2 Decide whether groups are better shown by colour within one panel or in separate panels (facets).
  • 3.1 Recognize common poor displays (a truncated axis, means without spread, a pie chart with many slices, overplotting and a dual axis), describe what each one conceals, and name a better display.

1.1 What Exploratory Data Analysis Is

Exploration is easiest to understand when it serves a concrete request. The case below introduces the request that runs through the whole lesson, a one-page visual briefing on social connection that an analyst prepares for a regional working group on loneliness, and it names the survey from which every plot is drawn.

📋 The running case: a visual briefing on social connection

A regional health authority has formed a working group on loneliness. Before it commissions any statistical modelling, the group has asked an analyst for a one-page visual briefing on social connection, drawn from the 2021 wave of the Canadian Social Connection Survey (CSCS). The group has three questions: how lonely people in the survey are, whether loneliness differs by age and gender, and how loneliness goes together with social support and mental health. The 2021 wave has 4,045 respondents, one row per person, and 3,083 of them answered every question used in the briefing. Each section of this lesson adds plots to the briefing, and Section 4 assembles them into a single multi-panel figure with a caption. Section 2 describes how the survey recruited its respondents and what its sample can support.

Before drawing anything for the working group, the analyst needs to be clear about what kind of work the briefing is. The statistician John Tukey set out this kind of work in his book Exploratory Data Analysis (Tukey, 1977). He distinguished two activities. Confirmatory analysis tests a question that was set before the data were examined, and it reports estimates, confidence intervals and p-values. Exploratory analysis asks what the data look like, using plots and simple summaries, and it is open to finding things that nobody planned to look for. Tukey compared exploration to detective work: the analyst gathers clues about the data before deciding which formal test or model to apply.

Exploration serves three purposes in an epidemiological analysis. The first is to check data quality. By this we mean looking for values that are impossible (such as an age of 250), implausible (such as a person who reports working more than 16 hours a day, every day) or suspiciously common (such as an unexpected crowd of people at the top of a scale). Values like these usually point to a recording error, a coding rule or a problem with the question, and they need to be resolved before any result can be trusted. The second purpose is to describe distributions, that is, how the values of each variable are spread out. The shape of a distribution decides how the variable should be summarized and used. For example, a mean is a fair summary of a roughly symmetric variable, whereas a median is better for a skewed one, because a few very large values pull the mean toward them. A strongly skewed or oddly shaped variable may also need to be transformed or grouped before it enters a model. The third purpose is to show relationships between variables. Here the analyst asks whether two variables rise together or move in opposite directions, whether the relationship follows a straight line or a curve, and whether it looks different in different groups, such as women and men. The answers decide how each variable should enter a model, for example as a straight-line term, as a curve or in groups. This lesson develops the basic plots for one and two variables (histograms, boxplots, bar charts and scatterplots) into an exploratory workflow and adds displays for many variables.

Table 6.1 sets out five questions that the analyst asks during exploration, what a plot can show in answer to each, and how the answer changes the later analysis.

Table 6.1. Questions asked during exploration, what a plot can show, and what the answer changes in the analysis.

Question the analyst asksWhat the plot can showWhat it changes in the analysis
What shape does each variable have?Symmetry or skew, one peak or two, a floor or ceilingThe choice between a mean and a median; transformation or grouping
Are there unusual or impossible values?Outliers, values outside the possible range, spikes at a maximumChecking the codebook; recoding values to missing; a sensitivity analysis
Do answers pile up on particular values?Heaping on round numbers or on the end of a scaleCaution in interpreting small differences; grouping the variable
How do two variables go together?Direction, form (straight or curved) and strengthHow each variable enters a model; whether a curve is needed
Does a pattern differ between groups?Different shapes or slopes in different panelsWhether an interaction or a stratified analysis is needed

Each question in Table 6.1 is answered by looking at a plot rather than by reading a single number. The next subsection shows, with four small datasets, why a summary statistic cannot take the place of that look.

1.2 Why Summary Statistics Alone Can Mislead

In 1973 the statistician Francis Anscombe published four small datasets to argue that graphs are an essential part of statistical analysis and that a computer should produce graphs as well as calculations (Anscombe, 1973). Each dataset has eleven pairs of values, labelled x and y, and the four datasets are supplied with R as a data frame called anscombe. Their summary statistics are almost identical, as Table 6.2 shows.

Table 6.2. Summary statistics for the four datasets in Anscombe's quartet (Anscombe, 1973).

StatisticSet 1Set 2Set 3Set 4
Mean of x9.09.09.09.0
Mean of y7.57.57.57.5
Standard deviation of x3.323.323.323.32
Standard deviation of y2.032.032.032.03
Correlation of x and y0.8160.8160.8160.817
Fitted liney = 3.00 + 0.500xy = 3.00 + 0.500xy = 3.00 + 0.500xy = 3.00 + 0.500x

A report that gave only these numbers would describe the four datasets as the same. The plots show that they are very different. Figure 6.1 draws each of the four sets as a scatterplot with its fitted line.

Four scatterplots labelled Set 1 to Set 4, each with the same red fitted line. Set 1 is a loose linear cloud; set 2 is a smooth arch; set 3 is a nearly perfect line with one high point; set 4 has ten points stacked at x equals 8 and one point at x equals 19.
Figure 6.1. Anscombe's quartet, with the least-squares line (red) added to each set. The line is the same in all four panels, and it describes only set 1 well. Data: Anscombe (1973), the anscombe data frame in R.

Each set in Figure 6.1 fails or passes for a different reason, and the reasons are the ones an analyst meets in real data. Expand each panel of the accordion below to read what the plot of that set shows and whether the fitted line is a fair summary of it.

Set 1: the situation a straight line was built for

The points scatter evenly around a straight line. A correlation of 0.82 and a fitted slope of 0.5 are fair summaries of these data, and the assumptions of linear regression look reasonable.

Set 2: a curve

The points follow a smooth arch with almost no scatter. The relationship is strong, but it is curved, so a straight line misdescribes it: the line overestimates y at both ends and underestimates it in the middle. A model with a squared term for x would fit these data almost exactly.

Set 3: one outlier

Ten points lie almost exactly on a line, and one point lies far above it. That single point pulls the fitted line upward and lowers the correlation from nearly 1 to 0.82. The analyst would check whether the point is a recording error before deciding how to handle it.

Set 4: one influential point

Ten of the eleven points share the same value of x (8), so within those points there is no relationship to estimate. A line describes how y changes as x changes, and among points that all have the same x there is no change in x from which to learn this. The one point at x = 19 therefore creates the whole correlation and decides the slope of the line on its own: if that point were moved up or down, the line would swing to follow it. In regression terms, the point has the highest leverage possible, so much influence that the fitted line passes exactly through it (R cannot even calculate a Cook's distance for such a point). Leverage describes how far a point lies from the other points along the x axis. By this we mean that a point with an unusual value of x has more power to tilt the fitted line than a point near the middle of the data, in the way that a long lever moves a load more easily than a short one. Cook's distance describes how much the fitted line would change if the point were removed. Here the point lies so far from the others along the x axis that the fitted line must pass through it, and without it the data would contain no information about the slope at all.

Taken together, the four panels show one fair summary, one wrong shape, one outlier and one influential point, all hidden behind the same numbers. The same lesson has been repeated with more recent examples. Starting from a dataset drawn in the outline of a dinosaur by Alberto Cairo, Matejka and Fitzmaurice (2017) generated the "Datasaurus Dozen", a set of further datasets that form stars, circles and lines, each with the same means, standard deviations and correlation as the dinosaur to two decimal places. The point is the same as Anscombe's: a summary statistic describes one feature of the data, and only a plot shows whether that feature is the one that matters. Because the quartet is supplied with R, this claim can be checked directly, which the next subsection does.

1.3 Anscombe's Quartet in R

Activity 6.1 reproduces Table 6.2 and Figure 6.1 from the anscombe data frame. It prints the means and standard deviations with colMeans() and sapply(anscombe, sd), the four correlations with cor() and the intercept and slope of each fitted line with coef(lm(...)), and it then draws the four scatterplots in one window with par(mfrow = c(2, 2)), adding the line y = 3 + 0.5x to each with abline(). Download the code and run it along with the text version below. As you run it, compare the console output for the four sets, which is almost identical, with the four plots, which are not; the questions then ask what an analyst who saw only the numbers would conclude, what each plot shows, and what to do about the single point that pulls set 3 away from its line.

R Activity 6.1: Anscombe's quartet in R

Text version of the activity: the code of the run-along script, its console output and notes on reading it.

anscombe                          # four small datasets that come with R
round(colMeans(anscombe), 2)      # the mean of each column
round(sapply(anscombe, sd), 2)    # the standard deviation of each column
Console output
x1 x2 x3 x4 y1 y2 y3 y4 1 10 10 10 8 8.04 9.14 7.46 6.58 2 8 8 8 8 6.95 8.14 6.77 5.76 3 13 13 13 8 7.58 8.74 12.74 7.71 4 9 9 9 8 8.81 8.77 7.11 8.84 5 11 11 11 8 8.33 9.26 7.81 8.47 6 14 14 14 8 9.96 8.10 8.84 7.04 7 6 6 6 8 7.24 6.13 6.08 5.25 8 4 4 4 19 4.26 3.10 5.39 12.50 9 12 12 12 8 10.84 9.13 8.15 5.56 10 7 7 7 8 4.82 7.26 6.42 7.91 11 5 5 5 8 5.68 4.74 5.73 6.89 x1 x2 x3 x4 y1 y2 y3 y4 9.0 9.0 9.0 9.0 7.5 7.5 7.5 7.5 x1 x2 x3 x4 y1 y2 y3 y4 3.32 3.32 3.32 3.32 2.03 2.03 2.03 2.03

The data frame has eleven rows and eight columns: x1 and y1 form set 1, x2 and y2 form set 2, and so on. colMeans() gives the mean of every column, and sapply(anscombe, sd) applies the standard deviation function sd() to every column. Every x column has a mean of 9.0 and a standard deviation of 3.32, and every y column has a mean of 7.5 and a standard deviation of 2.03. In row 3, set 3 has y = 12.74, well above its neighbours.

round(cor(anscombe$x1, anscombe$y1), 3)   # correlation in set 1
round(cor(anscombe$x2, anscombe$y2), 3)   # set 2
round(cor(anscombe$x3, anscombe$y3), 3)   # set 3
round(cor(anscombe$x4, anscombe$y4), 3)   # set 4
coef(lm(y1 ~ x1, data = anscombe))        # intercept and slope, set 1
coef(lm(y3 ~ x3, data = anscombe))        # set 3
coef(lm(y4 ~ x4, data = anscombe))        # set 4
Console output
[1] 0.816 [1] 0.816 [1] 0.816 [1] 0.817 (Intercept) x1 3.0000909 0.5000909 (Intercept) x3 3.0024545 0.4997273 (Intercept) x4 3.0017273 0.4999091

The four correlations are 0.816, 0.816, 0.816 and 0.817. lm(y1 ~ x1, data = anscombe) fits a straight line, and coef() prints its intercept and slope: about 3.00 and 0.500 in sets 1, 3 and 4 (set 2 gives the same line).

par(mfrow = c(2, 2))    # four plots in one window: two rows, two columns
plot(anscombe$x1, anscombe$y1, main = "Set 1", xlab = "x", ylab = "y")
abline(3, 0.5, col = "red")       # add the line y = 3 + 0.5x
plot(anscombe$x2, anscombe$y2, main = "Set 2", xlab = "x", ylab = "y")
abline(3, 0.5, col = "red")
plot(anscombe$x3, anscombe$y3, main = "Set 3", xlab = "x", ylab = "y")
abline(3, 0.5, col = "red")
plot(anscombe$x4, anscombe$y4, main = "Set 4", xlab = "x", ylab = "y")
abline(3, 0.5, col = "red")
par(mfrow = c(1, 1))    # back to one plot per window
Four scatterplots labelled Set 1 to Set 4, each with the same red line y = 3 + 0.5x. Set 1 is a loose cloud around the line; set 2 is a smooth arch; set 3 is a nearly perfect line with one point far above it; set 4 has ten points stacked at x = 8 and one point at x = 19.

These lines print nothing to the console. They draw four scatterplots in the Plots pane, arranged in two rows and two columns by par(mfrow = c(2, 2)), and abline(3, 0.5, col = "red") adds the line y = 3 + 0.5x to each one, as in Figure 6.1. Section 2 explains par(mfrow = ...) in more detail.

Files for this activity Code Answer key

R Reflect on what you just ran

Answer each question from your own output, as on the final exam: read the numbers, describe what the plots show, and decide what to do about an unusual point.

1. Using your console output, list the statistics that the four sets share. Interpret the correlation of 0.816 in one sentence. What would an analyst who saw only these numbers conclude about the four sets?

Model answerAll four sets have a mean of 9.0 for x and 7.5 for y, standard deviations of 3.32 for x and 2.03 for y, a correlation of 0.816 (0.817 in set 4) and the same fitted line, y = 3.00 + 0.500x. A correlation of 0.816 describes a strong positive straight-line association: larger values of x tend to go with larger values of y, although the points do not lie exactly on a line. An analyst who saw only these numbers would conclude that the four sets show the same strong, positive, straight-line relationship, with y rising by about 0.5 for each one-unit increase in x.

2. Describe the pattern in each of your four plots. For which set is the line y = 3 + 0.5x a fair summary? What do the plots show that the correlation of 0.816 does not?

Model answerSet 1 is a cloud of points scattered evenly around a rising straight line; set 2 is a smooth arch that rises and then falls, so its relationship is strong but curved. Set 3 lies almost exactly on a straight line except for one point far above it (row 3); in set 4, ten points share the value x = 8 and a single point at x = 19 creates the whole relationship. The line is a fair summary only for set 1. The plots show what the correlation cannot: the form of each relationship and the single points that drive it, because Pearson's r measures only how closely the points follow a straight line; set 2 would need a curve in a later model, such as a squared term for x.

3. Run your own code: at the end of the script, type coef(lm(y3 ~ x3, data = anscombe[-3, ])) and cor(anscombe$x3[-3], anscombe$y3[-3]) and run them. They refit set 3 without row 3, the point far above the line. What does that one point do to the slope and the correlation? Should it be removed from the analysis, and what would you do next?

Model answerWithout row 3, the line for set 3 is y = 4.01 + 0.345x and the correlation is 0.9999966, almost exactly 1; this one point raises the slope from 0.345 to 0.500 and lowers the correlation from almost perfect to 0.816. It should not simply be removed because it does not fit: the analyst first checks it against the source records and the codebook for an entry error. An error is corrected or set to missing; a genuine value is kept, and the results are reported with and without it as a sensitivity analysis, with the decision stated in the methods. This is why an analyst looks at plots before fitting a model: identical summary statistics can hide a curve, an outlier or a single influential point, and only a plot shows whether the planned model suits the data.
Saved.

Your output and plots make the point of the quartet concrete: the statistics agree to two decimal places, and only the plots show that the line y = 3 + 0.5x suits set 1 alone. Since a plot is needed before any summary can be trusted, the analyst must next decide which plot to draw for each question.

2.1 Matching the Display to the Question

Choosing a display begins with two questions about the request: how many variables it involves, and what type each variable is. Variables fall into two broad types. A numeric variable records an amount and is continuous (any value in a range, such as age) or discrete (whole numbers, such as the number of people in a household). A categorical variable places each person into a group and is binary, ordinal (ordered groups, such as self-rated mental health from poor to excellent) or nominal (unordered groups, such as province). Scale scores such as the loneliness score, which takes the whole-number values 3 to 9, are numeric for plotting purposes, although their small number of distinct values affects how they look, as later sections show.

Lesson 2 already matched the basic plots to these two questions. The recall below restates that guide, and its retrieval question asks you to apply it to the working group's question about age before you read on.

Recall: Lesson 2, Section 3 (Visualization Techniques)

Lesson 2, Section 3 matched the basic plots to the question and the variable type. A histogram shows the distribution of one numeric variable, a bar chart shows the frequencies of a categorical variable, boxplots compare a numeric variable across groups, and a scatterplot shows the relationship between two numeric variables. This section extends that guide to pairs of categorical variables and to many variables at once.

Retrieval question. Which display would you draw to see whether loneliness scores differ between four age groups, and why?

AnswerBoxplots side by side, one for each age group. Loneliness is numeric and age group is categorical, so boxplots compare the centre and the spread of the scores across the groups, and showing the individual points as well reveals how many people are in each group.

Figure 6.2 extends that guide into a decision tree, which starts from the number of variables in the question and then branches on their types.

How many variables does the question involve? One variable Two variables Many variables NumericHistogram or boxplot CategoricalBar chart of counts Numeric and numericScatterplot Categorical and numericBoxplots with points Two categoricalBar chart of percentages Several numericScatterplot matrix or heatmap Any display, by groupFacets (small multiples)
Figure 6.2. The number of variables and their types point to a display. Sections 2 and 3 draw each of these displays in R, and Section 4 covers the displays for many variables.

A histogram divides the range of a numeric variable into intervals (bins) and draws a bar for the number of people in each one, so the bars touch and their order follows the number line. A bar chart draws one bar for each category of a categorical variable, with gaps between the bars, and the categories can be placed in any sensible order, such as the order of an ordinal scale or from most to least common. Students often confuse the two because both use bars. The test is the variable on the horizontal axis: a histogram has a numeric axis, and a bar chart has categories.

For two variables, the display follows the pair of types. A scatterplot places one point per person for two numeric variables. Boxplots side by side compare a numeric variable across the groups of a categorical variable, and adding the individual points shows how many people are in each group and how their values are spread. For two categorical variables, a bar chart of percentages (for example, the percentage of each gender in each mental health category) compares the groups fairly when the groups differ in size. By this we mean that each bar shows the share of a group that gave an answer, rather than the number of people who gave it. This matters because raw counts mix two things together: the size of the group and the pattern of answers within it. For example, there are far fewer non-binary respondents than women or men in the briefing file, so in a bar chart of counts every bar for non-binary respondents would be short, even if they rated their mental health in much the same way as everyone else. Converting the counts to percentages within each gender removes the difference in group size, so the bars answer the question that was actually asked, which is whether the pattern of ratings differs between the groups. For many variables, a scatterplot matrix or a correlation heatmap gives an overview of every pair, and facets repeat one display for each group in a grid of small panels, an idea that Tufte (1983) called small multiples.

Interactive 6.1 lets you practise the decision tree in Figure 6.2 on six requests from the working group. For each request, name the number and the type of the variables first, then click the display you would draw; the feedback under the options explains why each choice suits the request or fails it, and nothing is scored.

📊 Interactive 6.1: Try it: choose a display for the briefing

Each request below comes from the working group. Choose the display you would draw first. Feedback appears under the options, and nothing is scored.

1. How are loneliness scores (a numeric score from 3 to 9) distributed across the 3,083 people in the briefing file?

2. How many people rated their mental health as poor, fair, good, very good or excellent?

3. Does loneliness (numeric) differ between the four age groups (categorical)?

4. Is higher social support (a numeric score from 1 to 7) associated with lower loneliness?

5. Do women, men and non-binary respondents differ in how they rate their mental health (both categorical)?

6. Which of six numeric briefing variables (loneliness, support, depression, anxiety, life satisfaction and age) go together?

Several of the requests in Interactive 6.1 compare groups, such as genders or age groups. Once the type of display is settled, such a request raises one more choice, which is how the groups are told apart.

2.2 Colour or Separate Panels for Groups

Many questions in the briefing compare groups: women, men and non-binary respondents, or four age groups. Whatever display is chosen, the groups can be told apart by colour within one panel, or the display can be drawn once per group in panels that share a scale, which are called facets, or small multiples. Which works better depends on how many groups there are, how much their values overlap and how different their sizes are, as the rule below sets out.

Colour or facets?

When a display must show groups, the analyst can give each group its own colour within one panel, or draw one panel per group (facets). Colour works well for two or three groups whose values overlap little, and it lets the reader compare the groups directly. Facets work better when there are more groups, when the points overlap heavily, or when the groups differ greatly in size, because each group gets its own space on a shared scale.

The briefing uses both. In Section 3, three genders are told apart by fill colour within each age group (Figure 6.18), because three groups of boxes side by side are easy to compare, whereas twelve combinations of gender and age group each get their own panel (Figure 6.17), because twelve overlapping clouds of points could not be separated by colour and the small non-binary groups would be hidden behind the large ones. Whether groups are shown by colour or by panel, a display of the right type can still mislead when it is drawn poorly, and the last subsection examines the most common faults.

3.1 Common Poor Displays and What They Conceal

3.1.1 Four Displays That Conceal the Data

Research on graphical perception, the study of how accurately people read values from a graph, has found that people judge positions along a common scale more accurately than lengths, angles or areas (Cleveland & McGill, 1984). By this we mean that a reader can tell quite precisely which of two points placed against the same axis is higher, but is much less precise when asked which of two pie slices is larger or how much bigger one circle is than another. This finding gives a practical rule for building displays. A good display encodes the comparison that matters as a position or a length on a shared axis, so that the reader makes the comparison with the least error. In addition, a good display shows the data themselves wherever possible rather than a single summary of them, because, as Anscombe's quartet showed, a summary can hide the very shape that matters. Four displays that break these principles are common in reports and presentations: a bar chart whose axis does not start at zero, bars of group means that hide the spread, a pie chart with many slices, and a scatterplot in which the points cover one another. Each was drawn from the briefing file, beside a better version.

The four tabs below hold Figure 6.3 (a truncated axis), Figure 6.4 (means without spread), Figure 6.5 (a pie chart with many slices) and Figure 6.6 (overplotting). Open each tab, compare the poor version on the left with the better one on the right, and read under each figure what the poor version conceals; the first tab also explains the dual axis, a related distortion.

Left: bar chart of mean loneliness by gender with the axis starting at 5.3, making the bar for men look a fraction of the others. Right: the same bars with the axis starting at zero, where they look similar.
Figure 6.3. Mean loneliness score by gender (women 5.74, men 5.36, non-binary respondents 5.93; n = 3,083). Left: the axis starts at 5.3. Right: the axis starts at zero.

What it conceals. A bar encodes a value by its length, so a bar chart whose axis starts above zero misrepresents every comparison. On the left, the bar for men looks about one tenth of the height of the bar for non-binary respondents, although the means are 5.36 and 5.93, a ratio of about nine to ten. The right panel starts the axis at zero, so the lengths are proportional to the values. A related distortion is the dual axis, in which two series share a plot with two different vertical scales, one on the left and one on the right. Because the analyst chooses each scale separately, either series can be stretched or squeezed at will. For example, by changing the range of the right-hand axis, the same two series can be made to look as if they rise together, cross at a dramatic point, or barely move relative to each other. The apparent crossing points and relative heights therefore depend entirely on how the two scales were chosen rather than on the data, and two separate panels, each with its own clearly labelled axis, are usually clearer.

Note on axis ranges. The zero rule applies to bars. A dot or line display may use an axis that covers the possible range of the scale (3 to 9 for this loneliness score), provided the range is labelled.

Left: four bars showing mean loneliness by age group, all between 5.4 and 6.0. Right: boxplots for the same groups with every person drawn as a faint dot, showing scores from 3 to 9 in every group.
Figure 6.4. Loneliness score by age group (n = 3,083). Left: bars of the group means. Right: boxplots with each person shown as a jittered, transparent dot.

What it conceals. The bars show four means between 5.39 and 6.01 and nothing else. They hide the spread within each group (scores from 3 to 9 in every group), the shape of each distribution, and the number of people in each group, which ranges from 366 people aged 65 and older to 1,079 people aged 16 to 29. Weissgerber and colleagues (2015) reviewed 703 research articles in leading physiology journals, found that bar graphs were the most common way of presenting continuous data (85.6% of the articles included at least one), and recommended displays that show the distribution of the data, particularly dot plots of the individual values when samples are small. The right panel shows that the differences between age groups are small compared with the variation within each group.

Left: a pie chart of province of residence with fourteen slices, many of them thin. Right: the same counts as a horizontal bar chart sorted from British Columbia, the largest, to Yukon, the smallest.
Figure 6.5. Province or territory of residence in the 2021 wave (n = 4,045, including 50 people who did not answer). Left: a pie chart with 14 slices. Right: a bar chart sorted by count.

What it conceals. A pie chart asks the reader to compare angles and areas, which people judge less accurately than positions along a common scale. With fourteen slices, several of them thin, it is hard to tell whether Manitoba or New Brunswick has more respondents, or to read the smaller territories at all. The sorted bar chart places every category on a common scale, so the order and the size of the differences are clear. A pie chart can be acceptable for two or three categories that make up a whole; for more categories, a bar chart is clearer.

Left: a scatterplot of social support against loneliness in which the points form seven solid horizontal rows. Right: the same data with jittered, transparent points and a red fitted line sloping downward.
Figure 6.6. Loneliness score against social support (n = 3,083). Left: one solid point per person. Right: points jittered vertically by up to 0.2, drawn with transparency, and a least-squares line (red) with its 95% confidence band.

What it conceals. The loneliness score takes only seven values, so 3,083 points fall on seven horizontal lines and lie on top of one another. The left panel cannot show where most people are, and a row with five people looks the same as a row with five hundred. The right panel adds a small random vertical shift to each point (jitter) and draws the points with transparency, so dense areas look darker. The fitted line makes the downward trend easy to see: people with more social support tend to report less loneliness.

In each tab, the poor display hides either the comparison that matters or the data themselves, and the better version puts the comparison on a common scale and shows the people behind it.

3.1.2 Smaller Faults in Draft Figures

The four tabs show displays of the wrong form. Draft figures also fail in three smaller ways, and a critique of a figure should look for these as well. The first is labelling: an axis titled with an R variable name (such as life_sat) or a single letter (such as "y"), or with no units, leaves the reader to guess what is shown, so each axis is labelled in words with its units or range. The second is colour that carries no information or that relies on telling red from green: bright fills that only repeat the labels on the axis add nothing, and about 8% of men of northern European ancestry cannot tell red from green, so one plain colour, or a colour-blind-safe palette (Section 3), is the fix. The third is missing context: no sample size, data source or period anywhere in the figure, which the caption supplies (Section 4). When you critique a figure, state each problem, say what a reader could get wrong because of it, and give the fix.

This section has explained why the analyst looks at the data before fitting a model, how the question and the variable types point to a display and how groups are best shown, and what common poor displays conceal. The key takeaways below gather these ideas, the knowledge check reviews them, and Section 2 draws the displays for the briefing file in base R.

Key Takeaways

  • 1.1 Exploratory data analysis is the step where you look carefully at the data before fitting any model. Using plots and simple summaries, you find the shape of each variable, any unusual or impossible values, and how the variables go together. Confirmatory analysis is different: it tests a question that was set before the data were examined, and it reports estimates, confidence intervals and p-values. Exploration comes first because it tells you whether the planned model suits the data, and Table 6.1 links what each plot can show to what it changes in the analysis. When a question asks you to classify an activity, ask whether the question was fixed in advance and answered with a formal estimate or test (confirmatory), or whether the analyst is looking to see what the data contain, for example by checking for impossible values (exploratory).
  • 1.2 Anscombe's quartet is the classic warning that summary statistics can hide very different data. Its four small datasets share almost the same means, standard deviations, correlation (about 0.82) and fitted line (y = 3.00 + 0.500x), yet one follows a straight line, one is a smooth curve, one is a near-perfect line pulled away by a single outlier, and in one a single point creates the whole relationship. A summary statistic describes one feature of the data, and only a plot shows whether that feature is the one that matters. So if a question gives you only a correlation or a fitted line and asks what you can conclude, the safe answer is that you cannot tell whether the line is a fair summary until you have seen a scatterplot, because a curve, an outlier or one influential point could give the same numbers.
  • 1.3 Activity 6.1 lets you see this for yourself in R. colMeans(anscombe), sapply(anscombe, sd), cor() and coef(lm(y1 ~ x1, data = anscombe)) give almost identical numbers for all four sets, while the four scatterplots drawn after par(mfrow = c(2, 2)) look completely different. The fitted line is a fair summary only for set 1: set 2 is curved, in set 3 one outlier distorts the slope, and in set 4 one point at x = 19 decides the line on its own. When you describe output like this, first list what the sets share, then say in one sentence what each plot shows, and finish by saying which summary the plots support and which they do not.
  • 2.1 Choosing a display starts with two questions: how many variables the question involves, and what type each one is (numeric or categorical). For one numeric variable, draw a histogram; for one categorical variable, draw a bar chart of counts, keeping ordered categories in their scale order. For two numeric variables, use a scatterplot; for a numeric variable across groups, use boxplots with the individual points; and for two categorical variables, use a bar chart of percentages within each group, so that groups of different sizes are compared fairly. For many numeric variables, a correlation heatmap or a scatterplot matrix gives an overview of every pair. A common trap is confusing a histogram with a bar chart because both use bars: check the horizontal axis, which is a number line for a histogram and a set of categories for a bar chart. On an exam, name the variable types first and let them lead you to the display, as the decision tree in Figure 6.2 does.
  • 2.2 When a display has to show groups, you can give each group its own colour within one panel, or draw a separate panel for each group (facets, also called small multiples). Colour works well for two or three groups whose values overlap little, because the reader can compare them directly on the same axes. Facets work better when there are many groups, when the points overlap heavily, or when the groups differ greatly in size, because each group then gets its own space on a shared scale and a small group is not hidden behind a large one. When a question describes the groups, count them and ask how much they overlap and how different their sizes are. In this lesson, for example, three genders are told apart by fill colour within each age group (Figure 6.18), while twelve combinations of gender and age group each get their own panel (Figure 6.17).
  • 3.1 A display of the right type can still mislead when it is drawn poorly, and each common fault hides something specific. A bar chart whose axis starts above zero exaggerates differences, because readers judge a bar by its length, so the axis of a bar chart starts at zero. Bars of group means hide the spread, the shape and the number of people in each group, and boxplots with the individual points show all three. A pie chart with many slices asks readers to compare angles, which they do poorly, so a bar chart sorted by count is clearer; overplotting hides where most people are, which jitter and transparency fix; and a dual axis makes any crossing depend on how the two scales were chosen, so two separate panels are clearer. Check the smaller faults too: axes titled with R names such as life_sat or a single letter, colour that only repeats the labels or relies on telling red from green, and no sample size or data source. When you critique a figure, state each problem, say what a reader could get wrong because of it, and give the fix; for a truncated axis, say how much the bars exaggerate the difference (in Figure 6.3, the bar for men looks about one tenth of the height of the bar for non-binary respondents, although the means of 5.36 and 5.93 are close).
Knowledge check: this section

1. A health unit's protocol states that it will test, with a regression model, whether weekly hours of recreational screen time are associated with depressive symptoms among teenagers. Before fitting the model, the analyst draws a histogram of every variable and finds that screen time has a long right tail, including 14 answers above 168 hours a week (the number of hours in a week). Which statement correctly describes these two stages of work?

This reverses the two stages. Using the protocol's variables does not make an analysis confirmatory; the histograms ask what the data look like, while the regression tests the question set in advance.
Exploratory analysis checks data quality and the shape of each variable, which tells the analyst whether the planned model suits the data; the regression that answers the protocol's question is confirmatory.
A question set in advance does not make exploration unnecessary. Answers above the number of hours in a week are impossible and must be checked, and the long tail affects how screen time should enter the model.
Exploration does not remove plausible values. Only values shown to be impossible, such as answers above 168 hours, are recoded after checking; the rest of the long right tail is real data.
Answer: the histograms are exploratory and the protocol's regression is confirmatory. The information needed is what each kind of analysis does: confirmatory analysis tests a question set before the data were examined and reports estimates, confidence intervals and p-values, while exploratory analysis asks what the data look like. Here the histograms check data quality (answers above 168 hours would mean more than every hour of the week, so they are impossible and must be checked) and the shape of screen time (a long right tail affects how it should enter the model), which tells the analyst whether the planned model suits the data. The tempting option about deleting the tail is wrong because exploration does not remove plausible values; only values shown to be impossible are recoded after checking. (Section 1, 1.1 What Exploratory Data Analysis Is, Table 6.1; Objective 1.1)

2. An analyst runs part of the code from Activity 6.1 and gets the output below. A colleague who sees only this output writes: "All four datasets show the same moderately strong straight-line relationship, so y = 3.00 + 0.500x can be reported for each." The four scatterplots show a cloud of points around a rising line (set 1), a smooth arch (set 2), a near-perfect line with one point far above it (set 3), and ten points stacked at x = 8 with one point at x = 19 (set 4). Which response is correct?

> round(colMeans(anscombe), 2)
 x1  x2  x3  x4  y1  y2  y3  y4
9.0 9.0 9.0 9.0 7.5 7.5 7.5 7.5
> round(sapply(anscombe, sd), 2)
  x1   x2   x3   x4   y1   y2   y3   y4
3.32 3.32 3.32 3.32 2.03 2.03 2.03 2.03
> round(c(cor(anscombe$x1, anscombe$y1), cor(anscombe$x2, anscombe$y2),
+         cor(anscombe$x3, anscombe$y3), cor(anscombe$x4, anscombe$y4)), 3)
[1] 0.816 0.816 0.816 0.817
Anscombe's quartet was built to refute this. Identical means, standard deviations and correlations can come from very different patterns, so only the plots show whether one line describes a dataset.
Set 1 is right, but in set 3 the single outlier is exactly what pulls the line away from the other points. One point can distort a slope, so it is checked before the line is trusted.
The four correlations are essentially the same, so set 4 does not have a stronger relationship. Its ten points share one x value and carry no information about a slope; a single point decides the line.
The summary numbers cannot tell the sets apart, so the plots decide: only set 1 is an even cloud around a straight line, while a curve, an outlier and one influential point shape the other three.
Answer: the line is a fair summary only for set 1. The output shows that the four sets share the same means (9.0 for x, 7.5 for y), standard deviations (3.32 and 2.03) and correlations (about 0.82), so the numbers alone cannot tell them apart; the plots supply the missing information. Reading each plot in turn: only set 1 is an even cloud around a straight line; set 2 needs a curve; in set 3 one point pulls the line upward and lowers r from nearly 1 to 0.82; and in set 4 the ten points at x = 8 carry no information about a slope, so the single point at x = 19 decides the line. The option that accepts set 3 is tempting, but a single outlier is exactly what distorts the slope there, so the analyst checks that point before trusting the line. (Section 1, 1.2 Why Summary Statistics Alone Can Mislead, Figure 6.1, and 1.3 Anscombe's Quartet in R, Activity 6.1; Objectives 1.2 and 1.3)

3. A school health survey of 1,500 students (1,200 in urban schools and 300 in rural schools) receives three requests: (i) How is the one-way commute time, in minutes, distributed? (ii) Does commute time differ between the four ways of travelling to school (walk, cycle, bus, car)? (iii) Does the way of travelling differ between urban and rural students? Which set of first displays follows the course's guide for matching a display to the question and the variable types?

Following Figure 6.2: one numeric variable takes a histogram, a numeric variable across groups takes boxplots with points, and two categorical variables with unequal groups take percentages within each group.
A bar of counts at each commute time does not bin a numeric variable as a histogram does, bars of means hide the spread within each group, and counts in (iii) mainly show that the urban group is larger.
A scatterplot needs two numeric variables, but way of travelling is categorical, so boxplots suit (ii). Pie charts compare angles, which people judge less accurately than positions on a common scale.
A pie chart does not suit a numeric variable, a combined histogram hides the four groups being compared, and counts coloured by area mainly reflect the larger urban group rather than how often each group uses each mode.
Answer: a histogram, boxplots with points, and a bar chart of percentages within each group. The information needed is the number of variables in each request and their types: commute time is numeric, way of travelling is categorical with four groups, and school type is categorical with two groups of very different sizes. Following Figure 6.2, one numeric variable calls for a histogram; a numeric variable across the groups of a categorical one calls for side-by-side boxplots, with the points showing the spread and the group sizes; and two categorical variables call for a bar chart of percentages within each group, because the 1,200 urban students would outnumber the 300 rural students in every bar of counts. The second option is tempting, but bars of means hide the spread within each group, and counts in request (iii) would mainly show that the urban group is four times larger. (Section 1, 2.1 Matching the Display to the Question, Figure 6.2; Objective 2.1)

4. An analyst will show body mass index against age in one figure, with groups told apart. In which of these situations is giving each group its own colour within one panel the better choice than separate panels (facets)?

Nine groups with heavily overlapping points and very different sizes are the situations in which separate panels work better, because nine colours on top of one another are hard to tell apart.
Forty schools are far too many groups for colour. Many groups call for separate panels, since readers cannot match forty colours to a legend.
Colour suits two or three groups of similar size whose points overlap little, so the reader can compare them directly on the same axes. Women and men here meet every condition.
Having only two groups is not enough. With very different sizes and heavy overlap, the 35 newcomers would be hidden under the long-term residents, so each group needs its own panel.
Answer: women and men, about 900 of each, whose points overlap little. The information needed is the number of groups, how much their points overlap and how different their sizes are, because the rule in 2.2 depends on them: colour suits two or three groups that overlap little, and separate panels suit many groups, heavily overlapping points or groups of very different sizes. Two groups of similar size with little overlap meet every condition for colour, so the reader can compare them directly on the same axes. The option with newcomers is tempting because it also has only two groups, but the 35 newcomers would be hidden under 2,000 overlapping points, so each group needs its own panel. (Section 1, 2.2 Colour or Separate Panels for Groups; Objective 2.2)

5. A colleague's draft bar chart shows the mean number of alcoholic drinks in the past week for three age groups: 6.1 (19 to 34), 6.4 (35 to 54) and 6.9 (55 and older). The vertical axis starts at 6.0. The colleague writes that the oldest group drinks "about nine times as much" as the youngest. Which critique is correct?

This is the misreading that a truncated axis produces. A bar encodes a value by its full length from zero; with the axis at 6.0 only the tops of the bars are drawn, so their lengths are not proportional to the means.
A bar encodes a value by its length, so the axis must start at zero for lengths to be proportional to the values; starting at 6.0 turns a modest difference into an apparent ninefold one.
The truncated axis is the main fault and is not acceptable. Pie charts are also less accurate than positions on a common scale, and they suit only a few categories that make up a whole, not separate means.
Making small differences look large is the distortion, not a benefit. Colours that only repeat the axis labels add nothing, and the axis should start at zero.
Answer: the truncated axis turns a difference of about 13% into an apparent ninefold difference, so the axis should start at zero. The information needed is the three means and where the axis starts. A bar encodes a value by its length, so with the axis at 6.0 the visible lengths are 6.9 − 6.0 = 0.9 and 6.1 − 6.0 = 0.1, a ratio of 9, whereas the true ratio of the means is 6.9 / 6.1 = 1.13. Starting the axis at zero makes the lengths proportional to the values, and because bars of means also hide the spread, shape and size of each group, boxplots with the points shown would be a better display still. The first option is the misreading that a truncated axis produces, and colours that only repeat the axis labels add nothing. (Section 1, 3.1.1 Four Displays That Conceal the Data, Figures 6.3 and 6.4; Objective 3.1)

6. A city health unit surveyed 1,500 adults who cycle at least once a month. A colleague reports that age and weekly minutes of cycling have a correlation of −0.25, gives the fitted straight line, and says that no plot is needed because a regression of cycling minutes on age will be fitted anyway. Which reply is correct?

Exploration comes before the confirmatory model, and Anscombe's quartet shows that one correlation and line can arise from a straight cloud, a curve, an outlier or one influential point, so a scatterplot shows which.
A regression fits the straight line it is given; its estimates, intervals and p-values describe that line and do not reveal the shape of the data. Exploration comes first to check whether the planned model suits the data.
A correlation of any size can hide a curve or an outlier. A weak r could come from a weak straight-line trend, a curve or a few unusual cyclists, and only a plot shows which.
In Anscombe's quartet the four datasets also share their means and standard deviations, so these do not describe the data fully. A scatterplot is still needed.
Answer: disagree, because exploration comes before the confirmatory model and only a plot shows whether a straight line suits the data. The information needed is what each kind of analysis does and what Anscombe's quartet showed. The regression is confirmatory: it answers a question set in advance with estimates, confidence intervals and p-values, and it fits the straight line it is given. Exploration comes first because it tells the analyst whether the planned model suits the data. In Anscombe's quartet, four datasets share the same means, standard deviations, correlation and fitted line, yet one is an even cloud, one is a curve, one is a line pulled by an outlier and one is a relationship created by a single point. A correlation of −0.25 could therefore come from a weak straight-line trend, from a curve or from a few unusual cyclists, and a scatterplot of cycling minutes against age shows which. The option that trusts the regression is tempting, but a model's estimates describe the line it was asked to fit and do not show the shape of the data; the last option fails because the four sets in the quartet also share their means and standard deviations. (Section 1, 1.1 What Exploratory Data Analysis Is and 1.2 Why Summary Statistics Alone Can Mislead; Objectives 1.1 and 1.2)
● Complete the quiz to continue.
Section 2 of 4

Quick Looks in Base R

⏱ Estimated time: 50 minutes
Lesson 6 · Section 2

Quick Looks in Base R

Base R draws a useful diagnostic plot with one line of code.

Base R plotting

Four familiar functions and one new one

FunctionWhat it drawsVariables
hist(x)HistogramOne numeric
barplot(table(x))Bar chart of countsOne categorical
boxplot(y ~ group)Boxplots side by sideNumeric across groups
plot(x, y)ScatterplotTwo numeric
pairs(data)Scatterplot matrixSeveral numeric

Recall: the first four were used in the Lesson 2, Section 3 R activity, and pairs() is new.

Preparing the data

One briefing file for every plot

  • The code loads the CSCS file and keeps the 2021 wave, which has 4,045 people.
  • Short names replace the long survey names, for example loneliness and support.
  • Bracket recoding sets "Presented but no response" to missing and builds four age groups.
  • na.omit() keeps the 3,083 people with complete data on all nine briefing variables.
One variable

A histogram and a bar chart

Left: histogram of loneliness scores from 3 to 9, highest at 5 and 6, with a small rise at 9. Right: bar chart of self-rated mental health, highest for very good.
Loneliness score (left) and self-rated mental health (right), n = 3,083.
Finding a problem

A spike at the largest value

Panel A: histogram of household size falling from 1 person toward 19, with a tall bar at 20. Panel B: histogram of hours worked per week with tall bars at 0, 35 and 40 and a thin tail to 140.
Household size (A) has 116 people at its top value of 20. Hours worked per week (B) piles up on 40 and stretches to 140.
Responding to the problem

Check, decide and report

Check

The codebook and the source questions show that 20 means 20 or more.

Decide

A count analysis sets 20 to missing, and a grouped variable absorbs the problem.

Report

The methods section states the problem, the decision and the number of people affected.

Two variables

Boxplots and a scatterplot

Panel A: boxplots of loneliness by age group, medians 5, 5, 6 and 6. Panel B: scatterplot of support and loneliness forming seven solid stripes. Panel C: the same scatterplot with jittered, transparent points showing a downward trend.
Loneliness by age group (A) and against social support, before (B) and after (C) jitter and transparency.
Layouts and saving

Many plots at once, and plots saved to files

CodeWhat it does
pairs(brief[, c(...)])It draws a scatterplot for every pair of the chosen variables.
par(mfrow = c(1, 2))It places the next plots in one row of two panels.
png("file.png", width, height, res)It sends the next plots to an image file.
dev.off()It closes the file, which saves it.
Carry forward

What to take into the next section

  • Base R draws a quick diagnostic plot for each kind of question.
  • Quick plots reveal problems such as spikes at a top value and heaping on round numbers.
  • Jitter and transparency reveal patterns when many points share the same values.

Overview

Section 1 explained why the analyst looks before modelling and how the question and the variable types point to a display. This section draws those displays for the survey data with base R, the plotting functions that are part of R itself, and reads what they show. It first prepares the briefing file that every later plot uses and asks whom a file of complete cases describes. It then reads histograms for data problems, including one that the cleaning lesson left behind, and reads boxplots that compare groups. Finally it turns to the code: the base R functions that draw each display, jitter and transparency for data that share a small number of values, and layouts of several plots saved to a file. Activity 6.2 carries out every step in R.

Learning Objectives

  • 4.1 Prepare an analysis file in base R with bracket recoding and na.omit(), and explain what keeping only complete cases means for whom the figures describe.
  • 4.2 Read a histogram for shape, unusual and impossible values, missing-value codes, ceilings, heaping and top-coding, and respond to each problem by checking, deciding and reporting.
  • 4.3 Read side-by-side boxplots (median, box, whiskers and separate points), compare groups, recognize skew, and explain why points beyond the whiskers are not removed simply for being there.
  • 5.1 Draw a histogram, a bar chart, side-by-side boxplots, a scatterplot and a scatterplot matrix in base R, with titles and axis labels, and say what each plotting function draws.
  • 5.2 Explain how jitter and transparency show data that share a small number of values, and what they leave unchanged.
  • 5.3 Arrange several base R plots in one window with par(mfrow = ...), save a plot to a file with png(), and explain why saving with code is better than exporting by hand.

4.1 Preparing the Briefing File

All the plots in this lesson use one analysis file, called brief. The code that builds it follows the pattern of the course's data cleaning and regression walkthroughs. It loads the CSCS file from the course's GitHub repository, keeps the 2021 wave (which gives one row per person), and gives short names to six variables. It then uses bracket recoding, in which a condition inside square brackets selects the rows to change, to set the answer "Presented but no response" to missing (NA) for gender and self-rated mental health, and to build four age groups from age in years. The function factor() with a levels argument fixes the order in which categories appear in tables and plots. Finally, na.omit() keeps the people who have no missing value on any of the nine briefing variables.

Table 6.3 lists the nine briefing variables, the survey measure behind each one, its possible values and the way it is treated for plotting.

Table 6.3. The nine variables in the briefing file.

Short nameMeasure in the surveyValuesType for plotting
lonelinessThree-item UCLA Loneliness Scale (Hughes et al., 2004)3 to 9, higher is lonelierNumeric (seven whole values)
supportMultidimensional Scale of Perceived Social Support, mean of 12 items (Zimet et al., 1988)1 to 7, higher is more supportNumeric
depressionPatient Health Questionnaire-2 (Kroenke et al., 2003)0 to 6, higher is more symptomsNumeric (seven whole values)
anxietyGeneralized Anxiety Disorder-2 (Kroenke et al., 2007)0 to 6, higher is more symptomsNumeric (seven whole values)
life_satLife satisfaction, a single item1 to 10, higher is more satisfiedNumeric (ten whole values)
ageAge in years16 to 100Numeric
age_groupFour groups built from age16 to 29, 30 to 44, 45 to 64, 65 and olderCategorical (ordinal)
genderGenderWoman, Man, Non-binaryCategorical (nominal)
mental_healthSelf-rated mental healthPoor, Fair, Good, Very good, ExcellentCategorical (ordinal)

Two cautions apply to every result drawn from this file. The first concerns the survey itself. The Canadian Social Connection Survey (CSCS) is a national online survey of social connection, loneliness and health among people living in Canada. Respondents to the 2021 wave completed an anonymous online questionnaire between April and June 2021. They were recruited online and were not drawn at random from a sampling frame, so the chance that any person in Canada took part is unknown, and the public file contains no survey weights that could correct for this. Two consequences follow for the briefing. Percentages and means describe these respondents and should not be read as the prevalence of loneliness in the Canadian population, because people who answer an online survey about social connection may differ from those who do not. Associations between variables within the sample, such as the link between social support and loneliness, are the more useful product. This is because an association compares people within the sample with one another, so it can still be informative when the sample as a whole is not typical of the population. For example, the survey may contain a higher share of lonely people than the population does, yet the tendency for people with more support to report less loneliness can still hold among the people who took part. Associations are not fully protected, however, because they also assume that taking part did not depend jointly on both variables. By this we mean that the chance of answering the survey should not depend on a particular combination of the two, such as people who are both lonely and poorly supported being especially likely, or especially unlikely, to respond. If it did, the people in the sample would show a different relationship between support and loneliness from the one in the population, and the association itself would be biased.

The second caution concerns the people whom na.omit() drops. The function prints nothing, so this cost is easy to overlook, and it is the one most often missing from a report.

Who is left out?

The last line of the preparation code in Activity 6.2 (line 33 of the script), c(kept = nrow(brief), left_out = nrow(data) - nrow(brief)), prints both numbers: R reports that 3,083 of the 4,045 respondents in the 2021 wave were kept and 962 were left out because they skipped at least one of the nine questions. Using a single complete-case file means that every panel of the briefing describes the same people, which makes the panels comparable. The cost is that the briefing describes only people who answered every question, and such people can differ from those who skipped questions. The briefing should state the number of people included and the reason others were excluded.

With the briefing file prepared and its limits stated, the analyst takes a first look at each variable, which is a plot of its distribution.

4.2 Reading a Histogram for Data Problems

4.2.1 Shape, Ceilings and the Order of Categories

A histogram is the first look at a numeric variable, and a bar chart the first look at a categorical one. Figure 6.7 shows the histogram of the loneliness score and the bar chart of self-rated mental health, both drawn in Activity 6.2 with the functions described in 5.1.

Left: base R histogram of loneliness scores, with bars for scores 3 to 9; the tallest bars are at 5 and 6, and the bar at 9 is taller than the bar at 8. Right: base R bar chart of self-rated mental health from Poor to Excellent; Very good is tallest and Poor is shortest.
Figure 6.7. The two plots drawn in Activity 6.2: a histogram of the loneliness score and a bar chart of self-rated mental health (n = 3,083).

Reading the histogram starts with its bins. The argument breaks = seq(2.5, 9.5, by = 1) sets the edges of the bins at 2.5, 3.5, 4.5 and so on, so each bar holds exactly one whole score. Without it, R chooses its own bins, which can merge two scores into one bar or leave empty gaps, and the picture of a variable with only seven values can change a good deal. The counts behind the bars, which table(brief$loneliness) prints in Activity 6.2, show that the histogram rises from 433 people at the lowest score of 3 to a peak of 755 at a score of 6, and then falls to 179 at a score of 8. The bar at 9, with 270 people, is taller than the bar at 8. A rise at the top of a scale is called a ceiling pile-up. By this we mean that a group of people chose the most extreme answer to all three items, and because the scale cannot record anything lonelier, everyone at or beyond that level receives the same score of 9. Some of these people may be far lonelier than others, but the scale cannot tell them apart, so differences among the loneliest respondents are hidden. The mean (5.57) is a little above the median (5), which reflects the longer upper tail. This happens because the mean uses the size of every value, so the higher scores in the upper tail, including the pile-up at 9, pull it upward, whereas the median depends only on the score of the person in the middle.

The bar chart is read in the order of its categories. The bars follow the order set by factor(), from Poor to Excellent, and the counts that table(brief$mental_health) prints in Activity 6.2 give their heights. Very good is the most common rating (1,024 people), followed by Good (928), and far fewer people rated their mental health as poor (230) or fair (467). If the levels had not been set, R would have placed the categories in alphabetical order (Excellent, Fair, Good, Poor, Very good), which hides the shape of an ordinal scale.

The questions behind these two readings apply to any plot of one variable, and they are worth keeping as a checklist.

A checklist for one-variable plots

For each variable, the analyst asks whether the distribution is symmetric or skewed, whether it has one peak or more, whether values pile up at the lowest or highest possible value (a floor or a ceiling), whether answers heap on round numbers, whether any value lies outside the possible range, and how many values are missing. Each answer is recorded in the analysis notes, because each one can change a later decision about summaries, recoding or models.

Two items on the checklist, values outside the possible range and answers that pile up on particular values, are the ones a numeric summary most often hides, as the next variable shows.

4.2.2 A Problem That Cleaning Left Behind: Top-Coding and Heaping

The household size variable, GEO_housing_household_size, records the number of people each respondent lives with, not counting the respondent, so 0 means living alone (503 people). The course's cleaning walkthrough used it to build a living-alone indicator, and the numeric summary that Activity 6.2 prints for it looks unremarkable: a median of 3, a mean of 4.46 and a maximum of 20. The histogram tells a different story. Panel A of Figure 6.8 shows that histogram, and panel B shows a second variable from the 2021 wave, the hours worked per week.

Panel A: histogram of household size; bars fall from 723 at 1 person to about 16 at 19, then a bar of 116 at 20. Panel B: histogram of hours worked per week in 2-hour bins; tall bars near 0, 35 and 40, smaller peaks at 20 and 30, and a long thin tail to 140 hours.
Figure 6.8. Panel A: number of people the respondent lives with (n = 3,427; 618 missing). Panel B: hours worked per week during the COVID-19 pandemic (n = 2,978; 1,067 missing), in two-hour bins. Both variables are from the 2021 wave before any recoding.

In panel A, the bars fall smoothly from one other person toward larger households, and then a bar of 116 people appears at exactly 20, the largest value in the data. The real distribution of household sizes does not rise again at its maximum, so the spike signals that something about the value 20 differs from the other values. Tracing the variable back to the survey explains it. Household size is the sum of eight questions that ask how many partners, children, grandchildren, parents, in-laws, siblings, roommates and other people the respondent lives with, and totals above 20 were recorded as 20. For 105 of the 116 people at 20, the eight answers added up to more than 20, and one total reached 180. Some of these answers are implausible, such as five grandchildren, five in-laws and seven siblings in one household, and they may reflect careless or mistaken responses. The summary statistics did not reveal this problem, and the histogram made it obvious. Recording every value above a limit as the limit itself is called top-coding, and its sign on a histogram is a bar at the largest value that is taller than the bars just below it, so the top value means "this much or more".

The analyst's response has three steps. The first is to check the codebook and the source questions, which is how the meaning of 20 was established. The second is to decide how the variable will be used. Used as a count, the value 20 cannot be trusted and can be set to missing with a bracket recode (Activity 6.2, question 1). Used as a grouped variable, such as living alone versus living with others, or 0, 1, 2 to 4 and 5 or more, the problem is absorbed by the top category and matters much less. The third step is to report the decision in the methods section, with the number of people affected, and, where the decision could change a result, to repeat the analysis both ways as a sensitivity analysis.

Panel B shows two further problems that the same plot can reveal. Answers to the question on hours worked per week heap on round numbers: 340 of the 2,978 people who answered reported exactly 40 hours, compared with one person at 39 and one at 41, and smaller peaks appear at 20, 30 and 35. Heaping of this kind, also called digit preference, usually reflects two habits. Many people do not know their exact hours and round them to a convenient number, and many people have a standard working week of 35 or 40 hours and report that figure even when their actual hours vary from week to week. Because of this, a reported value of 40 stands for a range of real values rather than exactly 40 hours. The consequence is that small differences in reported hours carry little information: a difference of one or two hours between two people, or between two groups, may reflect nothing more than how each person chose to round. The long right tail reaches 140 hours, and 13 people reported more than 112 hours a week, which would mean working more than 16 hours a day, every day. These values are implausible and would be checked and, in most analyses, set to missing.

4.2.3 Missing-Value Codes and Narrow Bins

A third problem can look similar on a histogram but has a different cause. Many survey files store a non-answer as a number that cannot be a real value, such as 99 for household size or 999 for hours worked, and a histogram shows such a missing-value code as a separate bar far from the rest. The numeric summary often gives a clue first. One clue is a maximum that is impossible, such as 999 hours in a week. Another is a mean that lies above the third quartile. In most variables the mean sits somewhere in the middle half of the data, between the first and third quartiles. When a few people carry a huge code such as 999, those values pull the mean far upward, because the mean adds up the size of every value. The median and the quartiles, by contrast, depend only on the order of the values, so a handful of extreme values at the top barely moves them. A mean that has risen above the third quartile is therefore a sign that a few very large values are present. The codebook confirms what the code means, and the code is then set to missing before any analysis.

Heaping and codes can be seen only when the bins are narrow enough. A bin two hours wide puts 39, 40 and 41 hours into one bar, so a pile-up at 40 disappears, and a few extreme values stretch the axis so that the bars for most people look tiny. Narrower bins, such as one bar per whole score or per hour, show both problems. In base R the bins are set with breaks, and in ggplot2 with binwidth.

Top-coding, heaping and missing-value codes each call for a different response, and treating one as another is a common error, so the rule below is worth committing to memory.

Three problems, three responses

A missing-value code is recoded to missing. A top-coded value is real but inexact, so it is kept and treated as "this much or more" (for example by using the variable in groups with a top group), and labels and the report say so. Heaping cannot be corrected, so small differences are interpreted with caution or the variable is grouped. In every case, the methods section reports the problem, the decision and the number of people affected.

Household size and working hours show that a histogram can reveal problems that summary statistics hide. The same reading of centre, spread and unusual values carries over to a numeric variable compared across groups, where the display is a set of boxplots.

4.3 Reading Boxplots Across Groups

A plot of two variables shows how the variables go together. Figure 6.9 shows the boxplots of loneliness by age group (panel A) and two scatterplots of loneliness against social support (panels B and C), all drawn in Activity 6.2. Panel A is read here; panels B and C show a problem of drawing rather than of data, and they are read in 5.2.

Panel A: boxplots of loneliness for four age groups; medians 5, 5, 6 and 6; the 16 to 29 box is narrow, from 5 to 6, with points drawn at 3, 8 and 9. Panel B: scatterplot of social support against loneliness with seven solid horizontal stripes. Panel C: the same data jittered and transparent, darker between support 4 and 6 and loneliness 5 and 6, with a visible downward trend.
Figure 6.9. Loneliness by age group (A), and loneliness against social support drawn with solid points (B) and with jittered, transparent points (C), n = 3,083.

A boxplot summarizes a numeric variable with five numbers. The thick line is the median. The box runs from the lower to the upper quartile, so it holds the middle half of the people, and its length is the interquartile range (IQR). The whiskers extend to the most extreme values that lie within 1.5 times the IQR of the box. By this we mean that R marks off a distance of one and a half box lengths beyond each end of the box, and each whisker stretches to the last actual value inside that distance. Any value beyond the whiskers is drawn as a separate point. This rule is a convention, introduced by Tukey, for flagging values that sit unusually far from the middle half of the data; it does not decide whether a value is an error. Table 6.4 gives the numbers behind panel A.

Table 6.4. The numbers behind the boxplots in panel A of Figure 6.9: the number of people, the whiskers, the quartiles and the median in each age group.

Age groupPeopleLower whiskerLower quartileMedianUpper quartileUpper whisker
16 to 291,07945567
30 to 441,06034569
45 to 6457834689
65 and older36634679

The two older groups have a median of 6 and the two younger groups a median of 5, and the box for people aged 45 to 64 is the widest, so loneliness varies most in that group. The box for people aged 16 to 29 is narrow, because most of them scored 5 or 6. Its whiskers therefore stop at 4 and 7, and the scores 3, 8 and 9 in this group are drawn as separate points (212 people in all). These points are flagged by the 1.5 × IQR rule, and they are ordinary answers on the scale. The reason is that the rule measures distance in units of box length. Because the box for this group is very short, one and a half box lengths is also a short distance, so the whiskers stop close to the box and perfectly possible scores of 3, 8 and 9 fall beyond them. With a variable that takes only a few whole values, the points beyond the whiskers say more about the narrowness of the box than about unusual people.

The same reading applies to any boxplot. To compare groups, read the medians (the centre), the heights of the boxes and the lengths of the whiskers (the spread), and then the shape. A distribution that is skewed to the right (a long upper tail) shows a long upper whisker and a short lower one, a median near the bottom of the box, separate points above the box only, and, when the mean is marked, a mean above the median. Counts such as the number of doctor visits or days without a meal are usually right-skewed, because they cannot fall below zero but can be large, and in a skewed distribution many people are drawn beyond the upper whisker. Deciding what to do with those people is where analyses most often go wrong.

Points beyond the whiskers are flagged, not wrong

These points are flagged by the 1.5 × IQR rule, not shown to be wrong. In a skewed distribution they are often ordinary, possible values, and they may be the people who most need describing. The analyst checks any that are impossible (check, decide and report, as for the household-size spike), keeps the rest, and chooses a summary or a model that allows for a long tail, rather than deleting people to make the plot look tidier.

Exploration often points to the model as well. A count outcome that piles up at zero, has a long right tail and spreads more in groups with higher medians calls for a count model rather than linear regression. Comparing the variance of a count with its mean gives a first look at overdispersion, which means that the counts vary more than a Poisson model allows. A Poisson model assumes that the variance of a count equals its mean. So when the variance is several times the mean, the counts are much more spread out than that model expects, and a negative binomial model, which allows the variance to be larger than the mean, may be needed. This first look ignores the predictors, however. Some of the extra variation may be explained once the predictors are in the model, so the comparison is only a warning sign, and the formal checks are made on the fitted model. A predictor whose groups have clearly different medians is a candidate for the model.

Every plot read so far, from the histogram of loneliness to these boxplots, was drawn with a single line of base R code, and the next subsection turns from reading the plots to drawing them.

5.1 Drawing the Standard Plots in Base R

Base R draws a quick diagnostic plot with one line of code and needs no extra packages, which makes it the analyst's quick diagnostic tool: its plots are drawn, read and set aside, and only the useful ones are later rebuilt as polished figures with ggplot2 (Section 3). Every plotting function accepts the same labelling arguments: main for the title, xlab and ylab for the axis labels. Every plot in the briefing should carry these labels from the start, because a plot without labels is easily misread, even by the person who drew it.

Four of the functions are familiar from Lesson 2. The recall below lists them with the briefing variables to which this section applies them, and its retrieval question asks why one of them needs a helper function.

Recall: Lesson 2, Section 3 (Activity 2.3: describe the course data and check the scales)

Activity 2.3 in Lesson 2, Section 3 drew four standard plots with base R: hist(x) for a histogram of one numeric variable, barplot(table(x)) for a bar chart of one categorical variable, boxplot(y ~ group, data = ...) for boxplots of a numeric variable across groups, and plot(x, y) for a scatterplot of two numeric variables. This section applies the same four functions to the briefing file: the loneliness score, self-rated mental health, loneliness by age group, and loneliness against social support.

Retrieval question. Why is barplot() given table(x) and not the raw variable?

Answerbarplot() draws one bar for each number it is given, so it needs counts. table(x) counts the people in each category, and barplot() then draws one bar per count.

Reading the code of these functions follows the same logic as reading the plots. In hist(brief$loneliness, breaks = seq(2.5, 9.5, by = 1)), the variable goes along the horizontal axis and breaks fixes one bar per score (4.2.1). In barplot(table(brief$mental_health)), the bars follow the order of the factor levels. In boxplot(loneliness ~ age_group, data = brief), the variable before the tilde runs up the vertical axis and there is one box for each group named after it. In plot(brief$support, brief$loneliness), the first variable goes along the horizontal axis and the second up the vertical axis, with one point per person.

One function is new in this lesson. The function pairs() draws a scatterplot matrix, a small scatterplot for every pair of the chosen variables, with the variable names on the diagonal. Each panel is a quick look at one pair, and the panel above the diagonal shows the same pair as the panel below it with the axes swapped. With 3,083 people and scores that take few values, the panels are crowded, so the matrix is best used to spot strong patterns or unexpected shapes, after which a single scatterplot or a correlation heatmap (Section 4) gives a clearer view. Figure 6.10 shows the scatterplot matrix that Activity 6.2 draws for loneliness, support, depression and age.

A four by four scatterplot matrix for loneliness, support, depression and age. Panels involving loneliness and depression show grids of points because both take whole values; panels with age show vertical spread across ages 16 to 100.
Figure 6.10. The scatterplot matrix drawn by pairs() for four briefing variables (n = 3,083). Each off-diagonal panel is a scatterplot of the two variables named in its row and column.

The panels of Figure 6.10 that involve loneliness or depression look like grids of dots, because both scores take only a few whole values. The same problem affects the single scatterplot in panel B of Figure 6.9, and the next subsection explains how to fix it.

5.2 Jitter and Transparency for Overplotted Points

Panel B of Figure 6.9 draws one solid point for each of the 3,083 people. Because the loneliness score takes only seven values, the points fall on seven horizontal lines and cover one another, a problem called overplotting, and the plot cannot show where most people are. Panel C applies two remedies. The function jitter() adds a small random amount to each value so that points sharing a value spread out a little, and col = rgb(0, 0, 0, 0.15) draws each point in black at 15% opacity, so that areas where many points overlap look darker. The function set.seed() fixes the random numbers, so the jittered plot is the same each time the code runs. Jitter changes only the picture. By this we mean that the random shift is applied to where each point is drawn, not to the values stored in the data, so any mean, correlation or model calculated afterwards uses the original scores. The shift is also kept small, less than half the gap between neighbouring scores, so each point stays closer to its true value than to any other value and the plot can still be read correctly. Panel C shows that most people have support scores between 4 and 6 and loneliness scores of 5 or 6, and that people with more support tend to report less loneliness.

Jitter and transparency make the scatterplots readable without altering a single value. With every display now drawn and readable, the last step is to arrange the plots and keep them.

5.3 Several Plots in One Window, and Saving Plots

The setting par(mfrow = c(rows, columns)) divides the plotting window into a grid, and each new plot fills the next cell, row by row. par(mfrow = c(1, 2)) places two plots side by side, and par(mfrow = c(2, 2)), used in the Anscombe activity, places four in a square. The setting stays in force until it is changed, so the code resets it with par(mfrow = c(1, 1)) afterwards.

Plots in the RStudio Plots pane disappear when the session ends. To keep a plot, the analyst sends it to a file. The function png("name.png", width = 1600, height = 800, res = 200) opens an image file 1,600 pixels wide and 800 pixels high, at a resolution of 200 pixels per inch, which prints at 8 by 4 inches. Every plot drawn afterwards goes into the file and does not appear on screen. The function dev.off() closes the file, which saves it in the working directory. The functions pdf() and jpeg() work in the same way for other formats, and the Export button in the Plots pane saves a plot by hand. Saving with code is better practice, because the code records exactly how the figure was made and remakes it in one step if the data change.

Before you run the activity for this section, you may find it helpful to watch the code run. The card below links to a narrated R walkthrough of the base R plots in this section and the ggplot2 plots in Section 3.

Learn to do this in R

Narrated R walkthrough: Data Visualization in Base R and ggplot2

The walkthrough runs the code from this section and the next one line at a time, draws each plot as the line runs, and explains every argument. It is a good place to start if the activities feel fast.

Open the walkthrough

Activity 6.2 carries out every step of this section in R. It loads the survey and builds the briefing file (4.1), draws the histogram and the bar chart and finds the spike in household size (4.2), compares loneliness across age groups with boxplots (4.3), draws the scatterplots with and without jitter and the scatterplot matrix (5.1 and 5.2), and saves two plots side by side to a file (5.3). Download the code and run it along with the text version below; its preparation code is the starting point for every later activity in this lesson.

R Activity 6.2: quick looks in base R

Text version of the activity: the code of the run-along script, its console output and notes on reading it.

load(url("https://raw.githubusercontent.com/jorgeandr3s/heal/main/cscs/public_data/CSCS2025_full_cleaned_deidentified_data_and_metadata.RData"))   # survey data, read from the web
data <- data[data$SURVEY_collection_year == 2021, ]   # the 2021 wave: one row per person
nrow(data)                                             # how many people?
Console output
[1] 4045

The file adds three objects to the environment, and the one called data holds the survey responses. The square brackets keep the rows from the 2021 wave and every column. The 2021 wave holds 4,045 people.

data$loneliness <- data$LONELY_ucla_loneliness_scale_score   # 3 to 9, higher = lonelier
data$support    <- data$PSYCH_zimet_multidimensional_social_support_scale_score   # 1 to 7
data$depression <- data$WELLNESS_phq_score                   # 0 to 6
data$anxiety    <- data$WELLNESS_gad_score                   # 0 to 6
data$life_sat   <- data$WELLNESS_life_satisfaction_num       # 1 to 10
data$age        <- data$DEMO_age                             # years
data$gender <- as.character(data$DEMO_gender)
data$gender[data$gender == "Presented but no response"] <- NA
data$gender <- factor(data$gender, levels = c("Woman", "Man", "Non-binary"))
data$mental_health <- as.character(data$WELLNESS_self_rated_mental_health)
data$mental_health[data$mental_health == "Presented but no response"] <- NA
data$mental_health <- factor(data$mental_health,
  levels = c("Poor", "Fair", "Good", "Very good", "Excellent"))
data$age_group[data$age < 30] <- "16 to 29"
data$age_group[data$age >= 30 & data$age < 45] <- "30 to 44"
data$age_group[data$age >= 45 & data$age < 65] <- "45 to 64"
data$age_group[data$age >= 65] <- "65 and older"
data$age_group <- factor(data$age_group,
  levels = c("16 to 29", "30 to 44", "45 to 64", "65 and older"))
brief <- na.omit(data[, c("loneliness", "support", "depression", "anxiety",
                          "life_sat", "age", "age_group", "gender", "mental_health")])
c(kept = nrow(brief), left_out = nrow(data) - nrow(brief))   # people kept and left out
Console output
kept left_out 3083 962

Each bracket recode changes only the rows that meet the condition inside the brackets. The first age-group line, for example, writes "16 to 29" into age_group for every person whose age is below 30, and factor() with levels fixes the order of the categories. na.omit() then keeps the people with no missing value on the nine briefing variables, and the last line prints how many were kept (3,083) and how many of the 4,045 respondents were left out (962), which the briefing should report.

summary(brief$loneliness)                  # a numeric summary first
table(brief$loneliness)                    # the counts the bars will show
hist(brief$loneliness, breaks = seq(2.5, 9.5, by = 1),   # one bar for each score
     main = "Loneliness (UCLA 3-item scale)",
     xlab = "Loneliness score (3 to 9)", ylab = "Number of people")
table(brief$mental_health)                 # the counts the bars will show
barplot(table(brief$mental_health), main = "Self-rated mental health",
        xlab = "Rating", ylab = "Number of people")
Console output
Min. 1st Qu. Median Mean 3rd Qu. Max. 3.00 4.00 5.00 5.57 7.00 9.00 3 4 5 6 7 8 9 433 421 689 755 336 179 270 Poor Fair Good Very good Excellent 230 467 928 1024 434
Histogram of loneliness scores from 3 to 9, one bar per score: the tallest bars are at 6 and 5, and the bar at 9 is taller than the bar at 8.

The scores run from 3 to 9, the median is 5 and the mean is 5.57. The most common scores are 6 (755 people) and 5 (689). The bar at 9, the highest possible score (270 people), is taller than the bar at 8 (179): a ceiling pile-up. The bar chart follows the order set by factor(), from Poor to Excellent: Very good is the most common rating (1,024 people), and far fewer people rated their mental health as poor (230) or fair (467).

summary(data$GEO_housing_household_size)   # people each respondent lives with
hist(data$GEO_housing_household_size, breaks = seq(-0.5, 20.5, by = 1),
     main = "Household size",
     xlab = "Number of people the respondent lives with", ylab = "Number of people")
sum(data$GEO_housing_household_size == 20, na.rm = TRUE)   # how many at the top value?
Console output
Min. 1st Qu. Median Mean 3rd Qu. Max. NAs 0.000 1.000 3.000 4.463 6.000 20.000 618 [1] 116
Histogram of household size: the bars fall from 1 other person toward 19, and then a taller bar appears at 20, the largest value.

The summary uses the full 2021 wave, in which 618 people did not answer the household questions, and nothing in it looks unusual. The histogram (above) rises again at 20, the largest value, and sum(... == 20, na.rm = TRUE) counts the 116 people there. The argument na.rm = TRUE tells sum() to skip missing values, which would otherwise make the result NA.

boxplot(loneliness ~ age_group, data = brief, main = "Loneliness by age group",
        xlab = "Age group", ylab = "Loneliness score (3 to 9)")
tapply(brief$loneliness, brief$age_group, median)   # the thick line in each box
tapply(brief$loneliness, brief$age_group, IQR)      # the length of each box
Console output
16 to 29 30 to 44 45 to 64 65 and older 5 5 6 6 16 to 29 30 to 44 45 to 64 65 and older 1 2 4 3
Boxplots of loneliness for four age groups: medians 5, 5, 6 and 6; the 16 to 29 box runs only from 5 to 6, with points drawn at 3, 8 and 9; the 45 to 64 box runs from 4 to 8.

The formula loneliness ~ age_group draws one box for each age group. tapply() applies median() and IQR() to the loneliness scores within each group: the medians (the thick lines) are 5, 5, 6 and 6, and the boxes are 1, 2, 4 and 3 points long.

plot(brief$support, brief$loneliness,                  # every person as one point
     xlab = "Social support (1 to 7)", ylab = "Loneliness score (3 to 9)")
set.seed(2021)                                         # the same jitter every time
plot(jitter(brief$support), jitter(brief$loneliness),  # nudge each point a little
     pch = 16, col = rgb(0, 0, 0, 0.15),               # small grey see-through dots
     xlab = "Social support (1 to 7)", ylab = "Loneliness score (3 to 9)")
pairs(brief[, c("loneliness", "support", "depression", "age")],
      pch = 16, col = rgb(0, 0, 0, 0.1))               # every pair in one grid
The same scatterplot with jittered, see-through points: the darkest areas are at support scores of 4 to 6 and loneliness scores of 5 and 6, and loneliness is higher where support is low.

These lines draw three plots and print nothing. The first shows seven solid stripes. The second (above) uses jitter(), small filled dots (pch = 16) and a see-through grey (rgb(0, 0, 0, 0.15), where the last number is the opacity), so the dense areas look darker; set.seed(2021) makes the random jitter the same each time. pairs() draws one panel for every pair of the four variables, with the variable names on the diagonal.

png("briefing_base_r.png", width = 1600, height = 800, res = 200)   # open a file
par(mfrow = c(1, 2))                       # one row, two plots
hist(brief$loneliness, breaks = seq(2.5, 9.5, by = 1),
     main = "A. Loneliness", xlab = "Loneliness score", ylab = "Number of people")
boxplot(loneliness ~ age_group, data = brief,
        main = "B. Loneliness by age group", xlab = "Age group", ylab = "Loneliness score")
par(mfrow = c(1, 1))                       # back to one plot per window
dev.off()                                  # close the file, which saves it
Console output
png 2
The saved file briefing_base_r.png: panel A, the histogram of loneliness, beside panel B, the boxplots of loneliness by age group.

Nothing appears in the Plots pane, because png() sent both plots to the file briefing_base_r.png in R's working directory (the image above). The message after dev.off() names the graphics device R returned to: this recording printed png 2, and in RStudio it reads RStudioGD 2 or null device 1, depending on whether other plots are open. It can be ignored.

Files for this activity Data Code Answer key

R Reflect on what you just ran

Answer each question from your own output, as on the final exam: read the plot, name the problem, and say what the analyst should do.

1. For each of your two histograms (loneliness and household size), name the problem it reveals that its summary did not make obvious. What should the analyst do about the 116 people at 20 before any analysis, and what should be reported?

Model answerThe loneliness histogram shows a ceiling: the bar at 9, the highest score the scale allows, holds 270 people, more than the 179 at 8, whereas the summary shows only a maximum of 9. The household-size histogram shows top-coding: a real distribution does not rise again at its maximum, yet 116 people sit at exactly 20, because totals above 20 were recorded as 20, so 20 means "20 or more", and the summary (median 3, mean 4.463, maximum 20) did not reveal it. Check: the codebook and the source questions confirm what 20 means, and sum() gives the number of people affected (116). Decide: used as a count, 20 cannot be trusted and is set to missing with a bracket recode (data$household <- data$GEO_housing_household_size, then data$household[data$household == 20] <- NA); used in groups with a top group (for example 0, 1, 2 to 4, and 5 or more), it is kept as "5 or more" and the problem matters much less. Report: the methods section states the problem, the decision and the number of people affected (116), and the analysis is repeated both ways as a sensitivity analysis if the decision could change a result.

2. Describe what the boxplots and the output of lines 55 and 56 show about the centre and the spread of loneliness across the four age groups. Several scores in the youngest group are drawn as points beyond the whiskers. Should these people be removed from the analysis? Explain.

Model answerThe thick line in each box is the median: it is 5 in the two younger groups (16 to 29 and 30 to 44) and 6 in the two older groups (45 to 64 and 65 and older), so older respondents tend to report a little more loneliness. The box holds the middle half of each group, and its length, the interquartile range, shows the spread: it is longest for people aged 45 to 64 (4 points, from 4 to 8), so their scores vary most, and shortest for people aged 16 to 29 (1 point, from 5 to 6), because most of them scored 5 or 6. In the three older groups the whiskers reach from 3 to 9, the whole scale, so the differences between the groups are small compared with the variation within each group. No: the points beyond the whiskers are flagged by the 1.5 × IQR rule, not shown to be wrong, and the youngest group's box is so narrow that ordinary scores of 3, 8 and 9 fall beyond whiskers that stop at 4 and 7. They are possible answers on the scale, so the analyst checks for impossible values, keeps these people and reports the decision; deleting them would make the plot tidier but would bias the comparison.

3. Run your own code: at the end of the script, type plot(jitter(brief$depression), jitter(brief$loneliness), pch = 16, col = rgb(0, 0, 0, 0.15)) and run it. Both scores take only seven whole values. Why are the points jittered and drawn partly transparent, and what does jitter leave unchanged? What does your plot show about how loneliness is related to depression? Why is it better to save a figure with code, as lines 69 to 76 do, than to export it by hand?

Model answerDepression, like loneliness, takes only seven whole values, so with solid points the 3,083 people would sit on top of one another on a grid of 49 positions (overplotting), and a position held by one person would look the same as one held by hundreds. Jitter adds a small random shift to where each point is drawn, so points that share a pair of values spread out a little, and the see-through grey (the last number in rgb(0, 0, 0, 0.15) is the opacity) makes crowded areas darker, so the plot shows where most people are. Jitter changes only the picture: the values in brief, and any correlation or model computed from them, stay the same. The plot shows that loneliness tends to be higher among people with more depressive symptoms: people with no symptoms cluster at the lowest loneliness score (3), the darkest areas at depression scores of 2 and 3 are at loneliness scores of 5 and 6, and at the top of the depression scale (6) loneliness piles up at 9, although people vary widely at every level. Saving a figure with code, as lines 69 to 76 do, is better than exporting it by hand, because the code records exactly how the figure was made and remakes it in one step if the data change.
Saved.

Your output reproduces the numbers read in this section, from the 962 people left out of the briefing file to the 116 households recorded at 20, and the questions ask you to apply check, decide and report to them and to explain what jitter changes and what it leaves alone.

This section has prepared the briefing file, read histograms and boxplots for data problems and group differences, and drawn, arranged and saved the base R plots behind those readings. The key takeaways below summarize these steps, the knowledge check reviews them, and Section 3 rebuilds the most useful plots as polished figures with ggplot2.

Key Takeaways

  • 4.1 Every plot in the briefing comes from one analysis file, brief, prepared in base R. Bracket recoding changes only the rows that meet the condition inside the square brackets (for example, setting the answer "Presented but no response" to NA), factor() with levels fixes the order of the categories, and na.omit() keeps only the people with no missing value on the chosen variables, the complete cases. A single complete-case file means that every panel describes the same people, which makes the panels comparable. The cost is that the figures describe only people who answered every question, and those people may differ from the ones who skipped some. So whenever you see na.omit(), read from the row counts that R prints before and after the step (for example with nrow()) how many people were kept and how many were dropped, and say so in the report: here 3,083 of 4,045 respondents were kept and 962 (24%) were left out for missing data.
  • 4.2 A histogram is your first look at one variable, and with bins narrow enough (one bar per whole score, set with breaks in base R or binwidth in ggplot2) it shows problems that a numeric summary hides. Look for skew; a pile-up at the top or bottom of a scale (a ceiling or floor, such as the 270 people at the loneliness maximum of 9); heaping on round numbers (340 people reporting exactly 40 hours of work); top-coding, where a bar at the largest value is taller than the bars just below it because every larger value was recorded as that value (the 116 households at 20, meaning "20 or more"); values outside the possible range; and a separate bar far from the rest, which may be a missing-value code such as 99 or 999. The numeric summary often gives the first clue, with an impossible maximum or a mean that sits above the third quartile. Respond to each problem in three steps: check the codebook and the source questions, decide how to handle it, and report the decision and the number of people affected in the methods. As 4.2.3 sets out, a missing-value code is recoded to NA, a top-coded value is real but inexact and is usually kept as "this much or more" (for example by grouping the variable), and heaping cannot be corrected, so small differences are interpreted with caution.
  • 4.3 A boxplot summarizes a numeric variable in a few numbers: the thick line is the median, the box runs from the lower to the upper quartile and so holds the middle half of the people (its length is the interquartile range, or IQR), the whiskers reach the most extreme values within 1.5 times the IQR of the box, and anything beyond them is drawn as a separate point. To compare groups, read the medians for the centre, the box heights and whisker lengths for the spread, and then the shape; in Figure 6.9, for example, the two older age groups have a median of 6 and the two younger groups a median of 5. Right skew shows as a long upper whisker and a short lower one, a median near the bottom of the box, separate points above the box only, and a mean above the median when the mean is marked. Points beyond the whiskers are flagged by the 1.5 × IQR rule, not shown to be wrong (in the narrow box for people aged 16 to 29, ordinary scores of 3, 8 and 9 are drawn as points), so check any that are impossible and keep the rest rather than deleting people to tidy the plot. If the outcome is a right-skewed count whose variance is several times its mean, say that this first look points to a count model, possibly a negative binomial model, rather than linear regression.
  • 5.1 Base R is the analyst's quick diagnostic tool: each standard plot takes one line of code, and you draw it, read it and move on. hist(x) draws a histogram of one numeric variable; barplot(table(x)) draws a bar chart, and the table() step is needed because barplot() draws one bar for each number it is given, so it needs counts; boxplot(y ~ group, data = ...) draws one box per group; plot(x, y) draws a scatterplot; and pairs() draws a scatterplot matrix, with one small scatterplot for every pair of the chosen variables. Give every plot a title and axis labels from the start with main, xlab and ylab, because an unlabelled plot is easily misread, even by the person who drew it. When a question shows a line of base R code, say which plot it draws and which variable goes where; in boxplot(loneliness ~ age_group, data = brief), for example, loneliness runs up the vertical axis and there is one box for each age group.
  • 5.2 When many people share a small number of values, such as a loneliness score with only seven possible values, their points are drawn on top of one another (overplotting), and a row of five people looks the same as a row of five hundred. Jitter adds a small random shift to each point so that points sharing a value spread out a little, and transparency (col = rgb(0, 0, 0, 0.15) in base R, or alpha in ggplot2) makes crowded areas look darker, so together they show where most people are. They change only the picture: the data, and anything calculated from them, stay exactly the same, and set.seed() makes the random jitter the same each time the code runs. When asked why a figure uses jitter and transparency, name the shared values that cause the overlap, and say that the fix changes where the points are drawn, not the data or any fitted line.
  • 5.3 par(mfrow = c(rows, columns)) divides the plotting window into a grid that fills row by row, so c(1, 2) puts two plots side by side and c(2, 2) puts four in a square; the setting stays in force until you reset it with par(mfrow = c(1, 1)). To keep a plot, send it to a file: png("name.png", width = 1600, height = 800, res = 200) opens the file, every plot drawn next goes into it rather than onto the screen, and dev.off() closes the file and saves it in the working directory. A common surprise is that nothing appears in the Plots pane after png(); that is expected, and getwd() tells you where the file went. Saving with code is better than clicking Export, because the code records exactly how the figure was made and remakes it in one step if the data change. The printed size follows from the pixels and the resolution, so a file of 1,600 by 800 pixels at 200 pixels per inch prints at 8 by 4 inches.
Knowledge check: this section

1. A community health survey has 2,500 respondents. The analyst builds the analysis file with dat <- na.omit(survey[, vars]), where vars names the eight variables used in the figures, and prints the output below. Which sentence should the report include?

> nrow(survey)
[1] 2500
> nrow(dat)
[1] 1900
> nrow(survey) - nrow(dat)
[1] 600
> round(100 * nrow(dat) / nrow(survey))                    # percentage kept
[1] 76
> round(100 * (nrow(survey) - nrow(dat)) / nrow(survey))   # percentage dropped
[1] 24
na.omit() removes whole rows that have a missing value on any chosen variable, so respondents are dropped. The figures describe only those with complete data, not all 2,500.
Keeping only complete cases can introduce bias rather than remove it, because people who skip questions can differ from those who answer them all. That is why the number excluded is reported.
na.omit() keeps only rows with no missing value on the chosen variables, so every figure describes the same complete-case group; the report states both the number included and the number excluded.
na.omit() does not fill in missing values; it drops every row that has one. No values are replaced, and respondents are lost from the figures.
Answer: the report states that 1,900 of 2,500 respondents (76%) with complete data are included and 600 (24%) were excluded. The information needed is what na.omit() does and the counts that R prints. It keeps only the rows with no missing value on the chosen variables, so the output shows that 1,900 respondents were kept and 600 were dropped, and its last two lines give these as 76% and 24% of the 2,500. Every panel then describes the same 1,900 people, and the report gives both numbers because people who skip questions can differ from those who answer them all. The option claiming the figures are free of bias is tempting, but keeping only complete cases can introduce bias rather than remove it, which is exactly why the number excluded is reported. (Section 2, 4.1 Preparing the Briefing File, Who is left out?; Objective 4.1)

2. In a survey of 800 students, the question on days of school missed in the past 30 days gives the output below. The codebook has not yet been checked. What do these clues suggest, and what should the analyst do?

> summary(dat$days_missed)
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.
  0.000   0.000   1.000   3.744   3.000  99.000
> sum(dat$days_missed == 99)
[1] 12
A value outside the possible range and a mean pulled above the third quartile are the two clues to a missing-value code, so the response is check, decide and report: confirm in the codebook, recode to missing and state the number affected.
A 30-day period cannot contain 99 missed days, so 99 is not a real answer. The few huge values also pull the mean above the third quartile, so the mean is not a typical value here.
A mean above the third quartile points to a few very large values, which is a long right tail, not left skew. With such values the median, not the mean, is the better summary.
Top-coding records real values above a limit as the limit, so the top value lies within the possible range. 99 lies outside the range of 0 to 30 days, so it is a code, not a real but inexact value.
Answer: the clues point to a missing-value code, which is checked in the codebook, recoded to missing and reported. The information needed is the possible range (0 to 30 days), the maximum (99), the number of students at 99 (12) and where the mean sits relative to the quartiles. A value above 30 cannot be a real answer, and a few huge values raise the mean (3.744) above the third quartile (3) while the median (1) and the quartiles are unaffected; these are the two clues described in 4.2.3. The response follows check, decide and report: confirm in the codebook what 99 means, set it to missing with a bracket recode, and state in the methods that 12 students were affected. The top-coding option is tempting, but top-coding records real values above a limit as the limit, whereas 99 lies outside the possible range, so it is a code and not a real but inexact value. (Section 2, 4.2.2 A Problem That Cleaning Left Behind: Top-Coding and Heaping, and 4.2.3 Missing-Value Codes and Narrow Bins; Objective 4.2)

3. Boxplots compare waiting times at three walk-in clinics, and the table gives the numbers behind each box (in minutes). A manager proposes deleting the patients drawn beyond the whiskers before the clinics are compared. Which reading of the boxplots, and which reply to the manager, is correct?

ClinicPatientsLower whiskerLower quartileMedianUpper quartileUpper whiskerPoints beyond the whiskers
North2105152030526 points, 55 to 95
South18010253550853 points, 90 to 120
East955101520354 points, 40 to 70
The typical wait is read from the median and the spread from the box, not from the number of separate points. The 1.5 times IQR rule only flags points; it does not show that they are errors.
Separate points above the box and a longer upper whisker are the pattern of right skew, not left skew. Points are not removed to make the boxes look symmetric.
Spread is read from the box length, the IQR, not from the number of points beyond the whiskers, and the boxes for North and South differ in length. Deleting flagged points would also remove real long waits.
The median gives the centre, the box length gives the IQR, and longer upper whiskers with points only above show right skew; flagged points are kept unless a check shows an impossible value.
Answer: South has the longest and most variable waits, all three clinics are right-skewed, and the points beyond the whiskers are kept. The information needed is the median (the centre), the quartiles (the box, whose length is the IQR) and the whiskers and separate points (the tails). South's median of 35 minutes is the highest, and its box, from 25 to 50 minutes (an IQR of 25 minutes), is longer than North's (15 to 30) and East's (10 to 20). In every clinic the upper whisker reaches further from the box than the lower one (for North, from 30 up to 52 minutes, against from 15 down to 5) and the separate points lie only above the box, which is the pattern of right skew expected for waiting times, which cannot fall below zero but can be long. The manager's proposal is tempting, but the 1.5 × IQR rule only flags points; long waits are possible values and may be the patients who most need describing, so the analyst checks for impossible values, keeps the rest and uses a summary that allows for a long tail. (Section 2, 4.3 Reading Boxplots Across Groups, Table 6.4; Objective 4.3)

4. In a survey of 2,400 adults, the number of close friends (whole numbers from 0 to 10) is plotted against a wellbeing score (whole numbers from 1 to 7). With solid points the plot is a regular grid of dots, and a combination held by hundreds of people looks the same as one held by a single person. The analyst redraws it with the code below. Which statement about the new plot is correct?

plot(jitter(dat$friends), jitter(dat$wellbeing), pch = 16, col = rgb(0, 0, 0, 0.15),
     xlab = "Number of close friends (0 to 10)", ylab = "Wellbeing score (1 to 7)")
jitter() used inside plot() changes only where each point is drawn. The columns in dat, and any correlation computed from them, are unchanged.
When many people share the same pair of values, jitter spreads their points slightly and transparency lets overlaps build up into darker areas, showing where most people are without altering the data.
Transparency removes no points. Each person is still drawn; overlapping see-through points simply build up into darker patches where many people share values.
The 0.15 in rgb() is the opacity, 15%, not a size. It makes each point see-through so that crowded combinations look darker.
Answer: jitter spreads points that share values and transparency makes crowded areas darker, while the data are unchanged. The information needed is what each argument does. Both scores take only a few whole values, so many of the 2,400 people share each pair of values and their points sit exactly on top of one another (overplotting). jitter() adds a small random amount to where each point is drawn, pch = 16 draws small filled dots, and the last number in rgb(0, 0, 0, 0.15) is the opacity (15%), so areas where many points overlap build up into darker patches that show where most people are. The first option is tempting, but jitter() is applied inside plot() to the positions drawn; the columns in dat, and any correlation or model computed from them, do not change. (Section 2, 5.2 Jitter and Transparency for Overplotted Points; Objective 5.2)

5. An analyst runs the code below. Afterwards, the RStudio Plots pane is empty. Which statement is correct?

png("clinic_waits.png", width = 1600, height = 800, res = 200)
par(mfrow = c(1, 2))
hist(dat$wait, main = "A. Waiting time", xlab = "Minutes", ylab = "Number of patients")
boxplot(wait ~ clinic, data = dat, main = "B. Waiting time by clinic",
        xlab = "Clinic", ylab = "Minutes")
par(mfrow = c(1, 1))
dev.off()
An empty Plots pane is expected. After png() opens a file, everything drawn goes to that file rather than the screen until dev.off() closes and saves it; this is not an error.
This reverses the order. In mfrow the first number is the number of rows and the second the number of columns, so c(1, 2) places the two plots side by side.
png() sends the drawing to a file of the given size and resolution, par(mfrow = c(1, 2)) sets one row of two panels, and dev.off() saves the file; the script records exactly how to remake the figure.
par(mfrow = c(1, 2)) divides the page into two panels, so the second plot fills the second panel rather than replacing the first. Both plots are saved in the one file.
Answer: the file holds a histogram beside a set of boxplots, 8 by 4 inches, and the code records how it was made. Reading the code line by line: png() opens an image file 1,600 by 800 pixels at 200 pixels per inch (1,600 / 200 = 8 and 800 / 200 = 4 inches), so everything drawn next goes to the file and not to the screen; par(mfrow = c(1, 2)) divides the page into one row and two columns; hist() draws a histogram of one numeric variable and boxplot(wait ~ clinic, ...) draws one box per clinic; and dev.off() closes the file, which saves it in the working directory. The stacked option reverses the order: the first number in mfrow is the number of rows. Saving with code is better than exporting by hand because the script records exactly how the figure was made and remakes it in one step if the data change. (Section 2, 5.1 Drawing the Standard Plots in Base R and 5.3 Several Plots in One Window, and Saving Plots; Objectives 5.1 and 5.3)

6. A provincial survey asked 2,400 students how many sugary drinks they had in the past week. The codebook says that answers above 21 were recorded as 21 and that 99 means "don't know". The analyst prints the output below. How should the two values be handled?

> sum(dat$drinks == 99)
[1] 96
> sum(dat$drinks == 19)
[1] 2
> sum(dat$drinks == 20)
[1] 4
> sum(dat$drinks == 21)
[1] 41
Both values come from the codebook, but they mean different things. 21 stands for a real answer of 21 or more, so setting it to missing would drop the heaviest drinkers.
A missing-value code stands for no answer, so 99 is set to missing; a top-coded value stands for a real answer at or above the limit, so 21 is kept as 21 or more, and both decisions are reported.
This reverses the two values. The codebook says 21 records answers above 21, which is top-coding, and 99 means don't know, which is a missing-value code, not a number of drinks.
99 means don't know, so it is not a number of drinks, and including it would inflate the mean. Values must be checked against the codebook before any summary is reported.
Answer: 99 is set to missing and 21 is kept as "21 or more", and both decisions are reported. The information needed is what the codebook says each value means and what the output shows. A missing-value code stands for no answer, so the 96 values of 99 are not numbers of drinks; they are set to missing with a bracket recode such as dat$drinks[dat$drinks == 99] <- NA. The 41 students at 21, far more than the 2 at 19 and the 4 at 20, are the sign of top-coding: these students did answer, and 21 means "21 or more", so they are kept, the top value is labelled that way or the variable is used in groups with a top group, and the methods state the number affected. The first option is tempting because both values come from the codebook, but setting the top-coded students to missing would drop the heaviest drinkers, whose answers are real. (Section 2, 4.2.2 A Problem That Cleaning Left Behind: Top-Coding and Heaping, and 4.2.3 Missing-Value Codes and Narrow Bins; Objective 4.2)
● Complete the quiz to continue.
Section 3 of 4

The Grammar of Graphics with ggplot2

⏱ Estimated time: 50 minutes
Lesson 6 · Section 3

The Grammar of Graphics with ggplot2

Every ggplot2 plot is built from the same parts, added in layers.

The layered grammar

Six parts of every plot

Data

A data frame with one row per person.

Aesthetics

Mappings from variables to x, y, colour or fill.

Geoms

The marks: bars, boxes, points and lines.

Scales

Rules that turn values into positions and colours.

Facets

Panels that repeat the plot for each group.

Theme

Fonts, grid lines and background.

The template

Data, a mapping, a geom and labels

Line of codePart of the grammar
ggplot(brief, aes(x = loneliness)) +Data and the aesthetic mapping
geom_histogram(binwidth = 1, fill = "#0B7B6B", colour = "white") +The geom, with fixed (set) colours
labs(title = "...", x = "...", y = "...")Labels
ggplot2 histogram of loneliness scores with teal bars.
Bars

geom_bar() counts; geom_col() draws values

Left: geom_bar chart of counts for self-rated mental health. Right: geom_col chart of mean loneliness for four age groups.
Left: R counts the rows in each category. Right: R draws the four means it was given.
Groups

A boxplot with every person shown

Boxplots of loneliness by age group with jittered transparent points for all 3,083 people.
Two layers: geom_boxplot() and geom_jitter() (n = 3,083).
A smoother

Loneliness across age, with a loess curve

Scatterplot of age against loneliness with a blue loess curve that dips near age 30, rises to a second high point near 55 and falls after 70, with a wide grey band at the oldest ages.
The loess curve dips near age 30 and rises to a second high point in the fifties; the band widens where data are sparse.
Facets

Small multiples with facet_grid()

Twelve panels of support against loneliness, rows for woman, man and non-binary, columns for four age groups, each with a downward line; the non-binary panels have few points and wide bands.
Rows: gender. Columns: age group. The non-binary panels hold between 6 and 26 people each.
Colour

A colour-blind-safe palette

Boxplots of loneliness by age group with three boxes per group filled orange for women, sky blue for men and green for non-binary respondents.
Gender mapped to fill, with three Okabe-Ito colours and theme_minimal().
Saving

ggsave() writes a figure at a chosen size

ArgumentWhat it controls
"loneliness_age_gender.png"The file name; the extension sets the file type.
pThe plot to save; without it, the last plot shown is saved.
width = 7, height = 4.5The size in inches.
dpi = 300The resolution in dots per inch, the usual standard for print.
Carry forward

What to take into the next section

  • A ggplot2 plot combines data, aesthetic mappings and geoms, with scales, facets and a theme.
  • Layers can be stacked, such as a boxplot with jittered points or a scatterplot with a smoother.
  • Groups are shown with colour or facets, using a palette that people with colour-vision deficiency can read.

Overview

Base R plots are quick to draw, and Section 2 used them to check the data. A figure for the working group's briefing, or for a manuscript, needs more care: consistent labels, readable colours, panels that share a scale, and a file of the right size and resolution. The ggplot2 package (Wickham, 2016) is the most widely used tool in R for such figures. It belongs to the tidyverse, a family of R packages for working with data. The walkthroughs and this lesson keep data preparation in base R, exactly as in Section 2, and ggplot2 is the tidyverse package that this lesson uses for figures. This section describes the grammar on which ggplot2 is built and how to read a figure's code in plain words. It then builds the core displays, separates the two geoms that draw bars, reads a scatterplot with a smoother, compares groups with facets, and makes a figure readable for every reader with a colour-blind-safe palette and a theme before saving it to a file.

Learning Objectives

  • 5.4 Describe the layered grammar of graphics in terms of data, aesthetics, geoms, scales, facets and themes, and say in plain words what each line of a ggplot2 figure's code does for the figure.
  • 5.5 Build a histogram, a bar chart, a boxplot with jittered points, and a scatterplot with a smoother in ggplot2.
  • 5.6 Distinguish geom_bar(), which counts rows, from geom_col(), which draws supplied values.
  • 5.7 Read a scatterplot with a loess or straight line: describe the pattern, say what the grey 95% confidence band shows, and say where the line deserves the least trust.
  • 5.8 Compare a display across groups with facet_wrap() and facet_grid(), choose between colour and facets, and say when a panel holds too few people to interpret.
  • 5.9 Apply a colour-blind-safe palette and a theme, add a second cue that survives greyscale printing, and save a figure with ggsave().

5.4 The Layered Grammar of Graphics

5.4.1 Six Parts of Every Plot

Leland Wilkinson described a grammar of graphics in which every statistical graphic is assembled from a small set of independent parts, in the way that a sentence is assembled from parts of speech (Wilkinson, 2005). By this we mean that a chart is not treated as a fixed type, such as "a bar chart" or "a scatterplot", but as a combination of separate choices: which data are used, which variables are placed on which axes or given which colours, what kind of mark is drawn, and so on. Changing one choice while keeping the others gives a different chart. For example, loneliness by age group drawn with boxes gives boxplots, and the same variables on the same axes drawn with jittered points show every person instead. Hadley Wickham adapted the idea into a layered grammar, in which a plot is built by adding layers one at a time, and implemented it as ggplot2 (Wickham, 2010). The practical benefit is that one template covers almost every display: once the parts are familiar, a histogram, a boxplot and a faceted scatterplot differ only in a line or two of code.

Figure 6.11 lists the six parts of a ggplot2 plot, each with an example taken from the briefing.

Data brief: one row per person (3,083 rows, 9 variables) Aesthetics aes(x = age_group, y = loneliness, fill = gender) Geoms geom_boxplot(), geom_jitter(), geom_smooth() Scales scale_fill_manual(values = c("#E69F00", "#56B4E9", "#009E73")) Facets facet_wrap(~ age_group) or facet_grid(gender ~ age_group) Theme labs(title = ..., x = ..., y = ...) + theme_minimal()
Figure 6.11. The six parts of a ggplot2 plot, each with an example from the briefing. The first three are needed for every plot; the last three have sensible defaults and are changed when needed.

The six flip cards below expand the parts named in Figure 6.11. Click each card to read what the part does and which code controls it; the cards also serve as a reference while you run Activity 6.3.

DataClick to explore
AestheticsClick to explore
GeomsClick to explore
ScalesClick to explore
FacetsClick to explore
Themes and labelsClick to explore

Of the six parts, the data, the aesthetics and the geoms are needed for every plot, while scales, facets and the theme have sensible defaults and are changed only when needed. The distinction between mapping a variable inside aes() and setting a fixed value outside it, explained on the Aesthetics card, is the one that matters most when reading code.

5.4.2 The ggplot2 Template

The parts are joined into a single block of code by one template, shown as Equation 6.1. The function ggplot() receives the data and the aesthetic mappings, and each further part is added with a plus sign at the end of the line before it:

The ggplot2 template
ggplot(data, aes(x = ..., y = ...)) +
  geom_...() +
  labs(title = ..., x = ..., y = ...) +
  theme_...()Eq 6.1

Three errors are common when the template in Equation 6.1 is first used. Expand the panel below to read each error, the message or symptom it produces, and its remedy.

Three errors that beginners meet first

The package is not loaded. The message could not find function "ggplot" means that library(ggplot2) has not been run in this session. The package is installed once with install.packages("ggplot2"), and loaded with library(ggplot2) in every new session.

The plus sign starts a line. R reads each line as a complete command when it can. If a line ends without a plus sign, R draws the plot so far and then treats the next line, starting with +, as a separate command, which gives an error. The plus sign therefore goes at the end of a line, never at the start of the next.

A fixed colour is put inside aes(). Writing aes(colour = "blue") maps the word "blue" as if it were a variable, so the points are drawn in a default colour with a legend that says "blue". A fixed colour belongs outside aes(), as in geom_point(colour = "blue").

Each of these errors comes from the structure of the template: the package must be loaded, each plus sign ends a line, and only variables go inside aes().

5.4.3 Reading a Figure's Code in Plain Words

Knowing the parts of a plot also makes it possible to read code written by someone else, which is how a figure is checked. You do not need to remember function names to read ggplot2 code: ask of each line what it adds to the picture. Table 6.5 reads one block of code in this way. The code draws loneliness against social support with a straight line for each gender, in one panel per age group, and the rest of this section builds each of its layers in turn.

Table 6.5. Reading a figure's code line by line in plain words.

Line of codeWhat it does for the figure
ggplot(brief, aes(x = support, y = loneliness, colour = gender))Uses the briefing file, puts support across the bottom and loneliness up the side, and gives each gender its own colour for points and lines, with a legend.
geom_jitter(height = 0.2, alpha = 0.15)Draws one point per person, nudged slightly up or down and drawn see-through, so people with the same scores do not hide one another and crowded areas look darker.
geom_smooth(method = "lm")Adds a straight line of average loneliness across support for each colour group, with a grey 95% confidence band around it (method = "loess" would draw a curve that can bend).
facet_wrap(~ age_group)Splits the figure into one panel per age group, on the same axes, so the groups can be compared side by side.
labs(x = "Social support (1 to 7)", y = "Loneliness score (3 to 9)", colour = "Gender")Replaces R names with plain axis titles that give the range, and titles the legend.
theme_minimal()Gives the figure a plain white background with light grid lines.

Reading the finished figure then follows the same order: what the axes show, what the colours and panels separate, what the lines and bands say about the average, and how many people sit behind each panel. With the template and this way of reading code in hand, the next subsection builds the displays that Section 2 drew in base R.

5.5 Building the Core Displays in ggplot2

5.5.1 One Variable

The histogram in Activity 6.3 maps loneliness to x and adds geom_histogram(binwidth = 1), which gives one bar per whole score, as breaks did in base R. The fill colour and the white bar outlines are set outside aes(), so they apply to every bar. Figure 6.12 carries the same information as the base R histogram in Section 2 (Figure 6.7), with labels that can be reused in the briefing.

ggplot2 histogram of loneliness scores from 3 to 9 with teal bars and white outlines on a grey background; the tallest bars are at 5 and 6.
Figure 6.12. Loneliness score drawn with geom_histogram(binwidth = 1) (n = 3,083).

A bar chart of a categorical variable is drawn with geom_bar(), which needs only an x aesthetic, such as mental_health, because it counts the people in each category. The left panel of Figure 6.13 shows the result for self-rated mental health, with the bars in the order of the factor levels, as in the base R bar chart. The right panel uses a second geom that also draws bars, and 5.6 explains how the two differ.

Left: geom_bar chart of the number of people in each self-rated mental health category, tallest for very good. Right: geom_col chart of mean loneliness score in four age groups, all between 5.4 and 6.0.
Figure 6.13. Left: geom_bar() counts the rows in each category of self-rated mental health. Right: geom_col() draws the four group means computed by aggregate(). The titles were added for this figure.

For one variable, then, the ggplot2 code differs from the base R code mainly in its structure: the data and the mapping come first, and the geom names the display.

5.5.2 A Numeric Variable Across Groups

A boxplot with jittered points uses two geoms on the same axes. geom_boxplot(outlier.shape = NA) draws the median, the quartiles and the whiskers for each age group and suppresses the separate outlier points, because geom_jitter(width = 0.2, height = 0.2, alpha = 0.1) already draws every person. The width and height arguments set how far each point may be nudged sideways and up or down, and alpha sets the opacity, from 0 (invisible) to 1 (solid). Figure 6.14 shows the result for the four age groups.

Boxplots of loneliness score for four age groups, overlaid with faint jittered points forming horizontal bands at each whole score; the band of points is thinnest for the 65 and older group.
Figure 6.14. Loneliness score by age group, with each person drawn as a jittered, transparent point (n = 3,083).

The combined display shows three things that bars of means would hide. The medians differ by one point (5 in the two younger groups and 6 in the two older groups). Every group includes people across the whole range of the scale, from 3 to 9. And the groups differ in size, from 1,079 people aged 16 to 29 to 366 aged 65 and older, which the density of the points makes visible.

The same two-layer pattern builds the fourth core display. For two numeric variables, geom_jitter() draws the people and a second layer, geom_smooth(), adds a line that follows their average; 5.7 explains how to read that line. First, though, the two bar charts in Figure 6.13 deserve a closer look, because the geoms behind them are easily confused.

5.6 Counting Rows or Drawing Values: geom_bar() and geom_col()

Two geoms draw bars, and they get their heights in different ways. geom_bar() counts the rows in each category, so it needs only an x aesthetic, such as mental_health, and it suits raw data with one row per person. geom_col() draws bars whose heights are supplied in the data, so it needs both x and y. To draw mean loneliness by age group, the means are first computed with the base R function aggregate(), which returns a small data frame with one row per age group, and geom_col() then draws those values, as in the right panel of Figure 6.13. The common mistake is to use geom_bar() on such a summary table: each group appears in only one row, so it counts one row per group and draws every bar at a height of 1. Before choosing, ask whether each row is a person to be counted or a group whose value has already been computed.

The right-hand panel of Figure 6.13 is the "means without spread" display from Section 1, and it is shown here only to illustrate geom_col(). For the briefing, loneliness by age group is better shown with boxplots and the individual points, as in Figure 6.14, so geom_col() is best kept for values that are themselves the point of the figure, such as one rate per region. The scatterplot with its smoother, the remaining core display, needs more care in reading.

5.7 Reading a Scatterplot with a Smoother

A scatterplot maps two numeric variables to x and y. Because the loneliness score takes only seven values, the points are jittered vertically (height = 0.2) and drawn with transparency. The layer geom_smooth() adds a curve that summarizes the average of y across the values of x, with a grey 95% confidence band. The band shows the uncertainty about where the average lies at each value of x. By this we mean that a different sample of people would give a somewhat different curve, and the band marks a range of plausible values for the average at each value of x, given the data in hand. It is not the spread of individual people, most of whom lie outside it. The band also widens where there are few people, because an average based on few observations is less certain. The argument se = FALSE draws the line without the band. With method = "loess", the curve is a loess smoother (locally estimated scatterplot smoothing), which fits many small weighted regressions along the axis and joins them. In other words, at each point along the horizontal axis it fits a line using only the nearby points, giving the closest ones the most weight, so the curve follows the local pattern of the data and can bend where that pattern changes. With method = "lm", the curve is a straight least-squares line, a single line fitted to all the points at once, so it cannot bend. When it draws the curve, ggplot2 prints the message `geom_smooth()` using formula = 'y ~ x', which only reports the formula it used and is not an error. Figure 6.15 adds a loess smoother to a scatterplot of loneliness against age.

Scatterplot of age from 16 to 100 against loneliness score, with jittered transparent points and a blue loess curve with a grey band. The curve starts near 6.4 at age 16, falls to about 5.3 near age 30, rises to about 6.0 between 50 and 60, and declines after 70, where the band becomes wide.
Figure 6.15. Loneliness score by age with a loess smoother and its 95% confidence band (n = 3,083). Points are jittered vertically.

The curve tells a story that a straight line would miss. Predicted loneliness is highest among the youngest respondents (about 6.4 at age 16), falls to about 5.3 around age 30, rises to about 6.0 between ages 50 and 60, and declines after 70 (about 5.4 at age 80). The band is narrow where most people are (ages 20 to 40) and widens at both ends, where the curve rests on few people: 48 respondents are under 20 and 29 are 80 or older. The ends of a smoother deserve the least trust, and a single respondent aged 100 sits at the far right. A smoother is a description of the data. It suggests that age might enter a later model as a curve or in groups, and it gives no reason, by itself, for the pattern.

The smoother summarizes one relationship across the whole briefing file. Whether a pattern differs between groups is a separate question, which facets answer by drawing one panel for each group.

5.8 Comparing Groups with Facets

Facets draw the same plot in one panel per group. facet_wrap(~ age_group) takes one categorical variable and wraps its panels into a grid, which suits a single grouping variable with several levels. All panels share the same axes by default, so heights and positions can be compared across panels. Figure 6.16 applies facet_wrap() to the histogram of loneliness, with one panel for each age group.

Four histograms of loneliness score in a two by two grid, one for each age group; the 16 to 29 and 30 to 44 panels have tall bars at 5 and 6, and the two older panels have shorter bars spread more evenly.
Figure 6.16. Loneliness score by age group with facet_wrap(~ age_group) (n = 3,083). The panels share both axes.

The shared vertical axis shows both the shape and the size of each group. The two younger groups, with more than 1,000 people each, have tall bars at 5 and 6. The two older groups have shorter, flatter histograms with more people at both ends of the scale, and they hold most of the ceiling pile-up seen in Section 2: in the 45 to 64 group, 111 people scored 9, as many as scored 6 (112), and in the 65 and older group the most common score is 3, the lowest possible (85 people). The older groups are therefore more divided, with more people at the least lonely and the most lonely ends of the scale. If the groups' shapes matter more than their sizes, facet_wrap(~ age_group, scales = "free_y") gives each panel its own vertical axis, at the cost of making heights incomparable across panels.

facet_grid(gender ~ age_group) takes two variables, the first for the rows and the second for the columns, and draws one panel for every combination. Figure 6.17 shows social support against loneliness, with a straight line in each panel.

A grid of twelve scatterplots, three rows for woman, man and non-binary and four columns for the age groups, each with a blue line sloping downward. Lines in the older age groups are steeper. The non-binary row has few points and wide grey bands.
Figure 6.17. Loneliness score against social support by gender (rows) and age group (columns), with a least-squares line and 95% confidence band in each panel (n = 3,083).

The lines slope downward in all twelve panels, so higher support goes with lower loneliness in every combination of gender and age group. The lines look steeper in the two older age groups: within the age groups as a whole, the slopes are about −0.40 and −0.54 points of loneliness per point of support in the two younger groups and about −0.76 and −0.84 in the two older groups. The table of counts printed in Activity 6.3 is the necessary companion to the grid. The non-binary panels hold between 6 and 26 people, so their lines and bands are very uncertain, and any difference they appear to show would need a much larger sample to assess. An exploratory figure should show small groups, and the text should say how small they are.

Table 6.6 gives these counts, the number of people behind each panel of Figure 6.17, as table(brief$gender, brief$age_group) prints them in Activity 6.3.

Table 6.6. Number of people in each panel of Figure 6.17, by gender and age group.

Gender16 to 2930 to 4445 to 6465 and older
Woman490477362251
Man564557204109
Non-binary2526126

Facets are one of the two ways of showing groups that Section 1 compared. The other is to map a variable to colour (for points and lines) or fill (for bars and boxes), which shows the groups within one panel. Colour lets the reader compare groups directly on the same axes, and it works well for two or three groups. Facets give each group its own panel, and they work better when there are many groups, when points overlap heavily, or when one group is much smaller than the others and would be hidden. The two can also be combined, with colour for one grouping variable and facets for another, as in the code read in Table 6.5. Once colour carries information, the colours themselves need care.

5.9 Colour-Blind-Safe Palettes, Themes and Saving a Figure

5.9.1 Palettes and Themes

A palette has to work for every reader. About 8% of men and 0.5% of women of northern European ancestry have some form of red-green colour-vision deficiency, so a palette that relies on telling red from green fails a sizeable part of any audience. The Okabe-Ito palette (Okabe & Ito, 2008) is a set of eight colours chosen to remain distinguishable for people with the common forms of colour-vision deficiency. Its first three colours after black, orange (#E69F00), sky blue (#56B4E9) and bluish green (#009E73), are used below with scale_fill_manual(). The viridis palettes, available in ggplot2 as scale_fill_viridis_d() for categories, are another safe choice, and they also remain readable when printed in greyscale. When a figure may be printed in black and white, or when readers may not separate even these colours, a second cue makes the groups distinguishable without colour: different point shapes (shape = group inside aes()), different line types (linetype = group), or one panel per group. Figure 6.18 maps fill to gender in boxplots of loneliness by age group, with the three Okabe-Ito colours and a minimal theme.

Boxplots of loneliness score for four age groups, with three boxes per group filled orange for women, sky blue for men and bluish green for non-binary respondents, on a white minimal theme with a legend titled Gender.
Figure 6.18. Loneliness score by age group and gender, with fill mapped to gender, three Okabe-Ito colours and theme_minimal(base_size = 12) (n = 3,083).

The theme controls the appearance of everything except the data. The default theme, theme_grey(), uses a grey panel with white grid lines. theme_minimal() removes the grey panel, theme_bw() adds a black border, and theme_classic() keeps only the axis lines. The argument base_size sets the size of all text in points, and 11 or 12 suits most journal figures. Labels are set with labs(), which also names the legend through the aesthetic it describes, as in fill = "Gender".

A colour-blind-safe palette with a second cue, and a plain theme with readable text, make a figure usable by every reader on screen. The last step is to save it to a file at the size at which it will be printed or shown.

5.9.2 Saving a Figure with ggsave()

The function ggsave() writes a ggplot2 plot to a file. Its first argument is the file name, and the extension sets the format: .png for most uses, .pdf or .svg for figures that must stay sharp at any size, and .tiff when a journal asks for it. The second argument is the plot object, here p, and without it ggsave() saves the last plot shown. The arguments width and height are in inches by default, and dpi sets the resolution, with 300 dots per inch the usual requirement for print. The size of the file in pixels is the width and the height multiplied by the dpi, and question 3 of Activity 6.3 has R work it out for the saved figure. Text size is relative to the physical size of the figure. By this we mean that base_size sets the text size in points for the figure at the width and height at which it is saved, and the text keeps that size only when the figure is shown at that size. If the figure is later shrunk to fit a much narrower space, such as a column of a page, everything in it shrinks by the same proportion, including the text. A plot saved at a large width with a small base_size will therefore have text that is too small when the figure is shrunk to fit a page, which is why a figure is best saved at the size at which it will be used.

Activity 6.3 practises the whole section. It draws every plot, from the histogram in Figure 6.12 to the coloured boxplots in Figure 6.18, prints the group means behind the geom_col() chart and the counts in Table 6.6, and saves the last plot with ggsave(). Download the code and run it along with the text version below, reading each layer as Table 6.5 does; the questions ask you to say in plain words what the layers do, to read the lines, bands and panel sizes, and to make the figure readable in black and white.

R Activity 6.3: the grammar of graphics with ggplot2

Text version of the activity: the code of the run-along script, its console output and notes on reading it.

brief <- read.csv("https://heal-learn.ca/r-activities/data/cscs_brief.csv")   # course data, read from the website
brief$age_group <- factor(brief$age_group,
  levels = c("16 to 29", "30 to 44", "45 to 64", "65 and older"))
brief$gender <- factor(brief$gender, levels = c("Woman", "Man", "Non-binary"))
brief$mental_health <- factor(brief$mental_health,
  levels = c("Poor", "Fair", "Good", "Very good", "Excellent"))
nrow(brief)                      # one row per person
# install.packages("ggplot2")    # once per computer, if not yet installed
library(ggplot2)
Console output
[1] 3083

The briefing file is the one built in Section 2: 3,083 people with no missing value on the nine briefing variables. A file read with read.csv() stores categories as text, so factor() with levels restores their order. ggplot2 is installed once on each computer with install.packages("ggplot2") and loaded with library(ggplot2) in every new session; if R reports could not find function "ggplot", run library(ggplot2) again.

ggplot(brief, aes(x = loneliness)) +               # data and mapping
  geom_histogram(binwidth = 1,                     # one bar per score
                 fill = "#0B7B6B", colour = "white") +   # set, not mapped
  labs(title = "Loneliness scores",
       x = "Loneliness score (3 to 9)", y = "Number of people")
ggplot(brief, aes(x = mental_health)) +
  geom_bar() +                                     # counts the rows
  labs(x = "Self-rated mental health", y = "Number of people")
means <- aggregate(loneliness ~ age_group, data = brief, FUN = mean)
means                                              # one row per age group
ggplot(means, aes(x = age_group, y = loneliness)) +
  geom_col() +                                     # draws the values given
  labs(x = "Age group", y = "Mean loneliness score")
Console output
age_group loneliness 1 16 to 29 5.492122 2 30 to 44 5.388679 3 45 to 64 6.006920 4 65 and older 5.633880
Histogram of loneliness scores from 3 to 9 with teal bars and white outlines; the tallest bars are at 5 and 6, and the bar at 9 is taller than the bar at 8.

Each plot starts with the data and the aesthetic mapping, and every further part is added with a plus sign at the end of the line before it. The fill and outline colours of the histogram are set outside aes(), so every bar is teal with a white outline; a fixed colour written inside aes() would be treated as a variable, drawn in a default colour and given a legend. geom_bar() counts the rows in each category (Very good is the most common rating), while geom_col() draws the four means computed by aggregate(), from 5.39 to 6.01. Used on the table of means, geom_bar() stops with the error stat_count() must only have an x or y aesthetic, and with only x mapped it would draw four bars of height 1.

set.seed(2021)                                     # the same jitter every time
ggplot(brief, aes(x = age_group, y = loneliness)) +
  geom_boxplot(outlier.shape = NA) +               # no separate outlier dots
  geom_jitter(width = 0.2, height = 0.2, alpha = 0.1) +   # every person
  labs(x = "Age group", y = "Loneliness score (3 to 9)")
Boxplots of loneliness score for four age groups, overlaid with faint jittered points in bands at each whole score; the medians are 5, 5, 6 and 6, the box for 45 to 64 runs from 4 to 8, and the 65 and older group has the fewest points.

The boxplot layer draws the median, the quartiles and the whiskers for each age group, and outlier.shape = NA stops it from drawing outlying people a second time, because geom_jitter() draws every person. width and height set how far each point may be nudged, and alpha = 0.1 makes the points nearly transparent; set.seed(2021) makes the jitter the same every time the code runs. The medians are 5, 5, 6 and 6, every group spans the whole scale from 3 to 9, and the 65 and older group (366 people) is the smallest.

set.seed(2021)
ggplot(brief, aes(x = age, y = loneliness)) +
  geom_jitter(height = 0.2, alpha = 0.1) +
  geom_smooth(method = "loess") +                  # a curve that can bend
  labs(x = "Age (years)", y = "Loneliness score (3 to 9)")
Console output
`geom_smooth()` using formula = 'y ~ x'
Scatterplot of loneliness score against age with jittered transparent points and a blue loess curve with a grey band: about 6.4 at age 16, about 5.3 near age 30, about 6 between 50 and 60, then falling, with the band widest at the oldest ages.

The message reports the formula that geom_smooth() used and is not an error. The loess curve is about 6.4 at age 16, dips to about 5.3 around age 30, rises to about 6 between 50 and 60 and falls after 70. The grey band is its 95% confidence band; it widens at both ends, where only 48 people are under 20 and 29 are 80 or older, so the ends deserve the least trust.

ggplot(brief, aes(x = loneliness)) +
  geom_histogram(binwidth = 1, fill = "#0B7B6B", colour = "white") +
  facet_wrap(~ age_group) +                        # one panel per age group
  labs(x = "Loneliness score (3 to 9)", y = "Number of people")
set.seed(2021)
ggplot(brief, aes(x = support, y = loneliness)) +
  geom_jitter(height = 0.2, alpha = 0.2) +
  geom_smooth(method = "lm") +                     # a straight line per panel
  facet_grid(gender ~ age_group) +                 # rows: gender; columns: age
  labs(x = "Social support (1 to 7)", y = "Loneliness score (3 to 9)")
table(brief$gender, brief$age_group)               # people in each panel
Console output
`geom_smooth()` using formula = 'y ~ x' 16 to 29 30 to 44 45 to 64 65 and older Woman 490 477 362 251 Man 564 557 204 109 Non-binary 25 26 12 6
Twelve scatterplots of loneliness against social support, three rows for woman, man and non-binary and four columns for the age groups, each with a downward straight line and grey band; the bands in the non-binary row are wide.

facet_wrap(~ age_group) draws one histogram per age group on shared axes, and facet_grid(gender ~ age_group) draws one panel for each combination of gender (rows) and age group (columns), here with a straight least-squares line (method = "lm") in each panel. The table gives the number of people behind each panel: the non-binary panels hold only 25, 26, 12 and 6 people.

p <- ggplot(brief, aes(x = age_group, y = loneliness, fill = gender)) +
  geom_boxplot() +
  scale_fill_manual(values = c("#E69F00", "#56B4E9", "#009E73")) +   # Okabe-Ito
  labs(x = "Age group", y = "Loneliness score (3 to 9)", fill = "Gender") +
  theme_minimal(base_size = 12)                    # a plain theme
p                                                  # show the plot
ggsave("loneliness_age_gender.png", p, width = 7, height = 4.5, dpi = 300)
Boxplots of loneliness score by age group with three boxes per group, filled orange for women, sky blue for men and bluish green for non-binary respondents, on a white minimal theme with a legend titled Gender.

Storing the plot as p lets the same object be shown on screen and saved. The three fill colours are from the Okabe-Ito palette. ggsave() writes loneliness_age_gender.png, 7 by 4.5 inches at 300 dots per inch (2,100 by 1,350 pixels), and prints nothing.

Files for this activity Data Code Answer key

R Reflect on what you just ran

Answer each question from your own output, as on the final exam: say in plain words what each layer does for the figure, read the lines, bands and panel sizes, and make the figure readable for every reader.

1. Say in plain words what geom_boxplot(outlier.shape = NA) and geom_jitter(width = 0.2, height = 0.2, alpha = 0.1) do for this figure, and why the points are jittered and drawn partly transparent. The bar chart drawn with geom_col() summarizes each age group by its mean: name two things that display hides and this figure shows.

Model answerThe boxplot layer, geom_boxplot(outlier.shape = NA), draws a box for each age group, with the median, the quartiles and the whiskers, and leaves out separate outlier dots because the jittered points already show everyone. The jitter layer, geom_jitter(width = 0.2, height = 0.2, alpha = 0.1), draws one point per person, shifted a little at random sideways and up or down, so that the many people who share a score do not sit on top of one another; the transparency makes crowded areas darker, and the shift changes only where the points are drawn, not the data. Compared with four means between 5.39 and 6.01, the figure shows the spread within each group: every age group has scores across the whole range from 3 to 9, and the box for 45 to 64 runs from 4 to 8. It also shows the size of each group, with far fewer dots for the 366 people aged 65 and older than for the 1,079 aged 16 to 29, and it shows the medians of 5, 5, 6 and 6.

2. Describe what the lines in the facet_grid(gender ~ age_group) figure show about social support and loneliness, and say what the grey band around each line shows. If the briefing shows this grid, what should its text say about the size of the non-binary panels, and why? Use the table of counts.

Model answerIn every panel the line slopes downward, so people with more social support tend to report less loneliness, in every combination of gender and age group, although people vary widely around the lines. The lines look steeper in the two older age groups than in the two younger ones. The grey band is the 95% confidence interval for the line: it shows the uncertainty about the average loneliness at each level of support, not the spread of individual people, most of whom lie outside it. The non-binary panels hold only 25, 26, 12 and 6 people, against hundreds of women and men in most panels, so their lines are very uncertain and their bands are wide. The briefing should show these panels, but its text should state how many people each one holds and draw no conclusion about differences for non-binary respondents from them.

3. Run your own code: at the end of your script, type p + aes(linetype = gender) + labs(linetype = "Gender") and c(width = 7, height = 4.5) * 300 and run them. Some readers of the working group's report have colour-vision deficiency, and the report may be printed in black and white. Explain why the script uses the three Okabe-Ito colours rather than the default colours, and what your first line adds for a black-and-white printout. Then state what the width, height and dpi arguments of ggsave() control, and what the output of your second line tells you about the saved file.

Model answerThe three colours come from the Okabe-Ito palette, which was designed so that people with the common forms of colour-vision deficiency, such as red-green deficiency, can still tell the groups apart; the default ggplot2 colours for three groups include a red and a green. Colour alone does not survive black-and-white printing here, because the orange and the sky blue print as almost the same grey. Your first line maps gender to the line type as well, so the boxes of the three genders have solid, short-dashed and long-dashed outlines and the legend shows both cues; this second cue lets readers tell the genders apart without colour. In ggsave(), width and height set the physical size of the figure in inches (7 by 4.5), and dpi sets the resolution in dots per inch (300, the usual print standard). Your second line multiplies the width and the height by the resolution, and its output shows that the saved file is 2,100 pixels wide and 1,350 pixels high, enough for a sharp print at 7 by 4.5 inches.
Saved.

Your plots and output reproduce Figures 6.12 to 6.18 and Table 6.6, so each reading in this section can be checked against your own screen. If any step felt fast, the narrated walkthrough introduced in Section 2 builds the same plots layer by layer; the card below links to it again.

Learn to do this in R

Narrated R walkthrough: Data Visualization in Base R and ggplot2

The walkthrough builds the ggplot2 plots in this section layer by layer, so that the effect of each line of code can be seen before the next is added.

Open the walkthrough

This section has described the grammar on which ggplot2 is built and used it to read a figure's code, to draw the briefing plots for one variable, for a numeric variable across groups and for two numeric variables, to compare groups with facets and colour, and to make and save a figure that every reader can use. The key takeaways below summarize these steps, the knowledge check reviews them, and Section 4 adds displays for many variables and combines the briefing plots into one figure.

Key Takeaways

  • 5.4 ggplot2 builds every plot from the same few parts, so once you know them you can read almost any figure's code. The data are a data frame with one row per person; aesthetics, written inside aes(), map variables to visual properties such as x, y, colour or fill; and geoms are the marks that draw the data, such as boxes, points or lines. Scales, facets and a theme have sensible defaults and are added as further layers when needed, each joined to the plot with a plus sign at the end of the line before it. To explain code in plain words, ask what each line adds to the picture, as Table 6.5 does: colour = gender gives each gender its own colour and adds a legend, and facet_wrap(~ age_group) makes one panel per age group on the same axes. Watch for the classic slip of putting a fixed colour inside aes(): aes(colour = "blue") treats "blue" as if it were a variable, whereas a fixed colour belongs outside, as in geom_point(colour = "blue").
  • 5.5 One template, ggplot(data, aes(...)) + geom_...() + labs(...) + theme_...(), draws every display in this section, and only the mapping and the geom change. geom_histogram(binwidth = 1) gives a histogram with one bar per whole score, geom_bar() a bar chart of counts, geom_boxplot(outlier.shape = NA) with geom_jitter(width = 0.2, height = 0.2, alpha = 0.1) a boxplot with every person shown as a faint dot, and geom_jitter() with geom_smooth() a scatterplot with a smoother. Because each geom is a layer, several can be stacked on the same axes; outlier.shape = NA simply stops the boxplot from drawing outlying people a second time when the jittered points already show everyone. If R reports could not find function "ggplot", run library(ggplot2), and keep each plus sign at the end of a line, never at the start of the next. When a question shows a block of ggplot2 code, be ready to say which display it produces and what each geom adds to it.
  • 5.6 geom_bar() and geom_col() both draw bars, but they get their heights in different ways. geom_bar() counts the rows in each category, so it needs only an x aesthetic and suits raw data with one row per person, such as the number of people giving each mental health rating. geom_col() draws heights that are already in the data, so it needs both x and y and suits a small summary table, such as the four age-group means computed with aggregate(). The tell-tale mistake is using geom_bar() on a summary table, which gives every bar a height of 1 because each group appears in only one row. Before choosing, ask whether each row is a person to be counted or a group whose value has already been computed, and remember that a chart of group means is the "means without spread" display from Section 1, so for the briefing, boxplots with points are better.
  • 5.7 geom_smooth() adds a line that follows the average of y across the values of x: method = "loess" draws a loess smoother, which can bend, and method = "lm" draws a straight least-squares line. The grey band around the line is its 95% confidence band, which shows how uncertain the position of the average is; it is not the spread of individual people, most of whom lie outside it, and se = FALSE draws the line without it. The line deserves the least trust where few people sit behind it, usually at the ends, and you can spot this because the band widens there (in Figure 6.15, only 48 respondents are under 20 and 29 are 80 or older). To describe such a plot, say the direction and form of the pattern in plain words (loneliness is highest among the youngest, dips around age 30, rises to about 6 in the fifties and falls after 70), note where the band is wide, and remember that a smoother describes the data and gives no reason for the pattern. The message `geom_smooth()` using formula = 'y ~ x' is only a note, not an error.
  • 5.8 facet_wrap(~ group) draws the same plot once for each level of one grouping variable and wraps the panels into a grid, and facet_grid(rows ~ columns) uses two variables, one for the rows and one for the columns, as in facet_grid(gender ~ age_group). The panels share their axes by default, so heights and slopes can be compared directly; scales = "free_y" gives each panel its own vertical axis, at the cost of that comparison. Use colour for two or three groups that overlap little, and facets for many groups, heavily overlapping points or groups of very different sizes. Always check how many people sit behind each panel, for example with table(brief$gender, brief$age_group): the non-binary panels in Figure 6.17 hold only 6 to 26 people, so their lines and bands are very uncertain. Show small panels rather than hiding them, but state how many people they hold and avoid drawing conclusions from them.
  • 5.9 About 8% of men and 0.5% of women of northern European ancestry have some form of red-green colour-vision deficiency, so a palette that depends on telling red from green fails part of any audience. Use a colour-blind-safe palette such as Okabe-Ito (for example scale_fill_manual(values = c("#E69F00", "#56B4E9", "#009E73"))) or viridis (scale_fill_viridis_d()), and if the figure may be printed in black and white, add a second cue that does not rely on colour: point shapes (shape = group), line types (linetype = group) or one panel per group. A theme such as theme_minimal(base_size = 12) sets the background and the text size without changing the data. Save the finished figure with ggsave("file.png", p, width = 7, height = 4.5, dpi = 300): the width and height are in inches, and 300 dpi is the usual print standard, giving a file of 2,100 by 1,350 pixels. When asked how to help readers with colour-vision deficiency, a full answer names both a safe palette and a second cue.
Knowledge check: this section

1. A colleague drew a figure of waiting times at three walk-in clinics with the code below. In plain words, what do the second and third lines do for the figure?

ggplot(dat, aes(x = clinic, y = wait)) +
  geom_boxplot(outlier.shape = NA) +
  geom_jitter(width = 0.2, height = 0, alpha = 0.2) +
  labs(x = "Walk-in clinic", y = "Waiting time (minutes)") +
  theme_minimal()
outlier.shape = NA stops the boxplot drawing its own points beyond the whiskers because the jitter layer already draws every patient; width nudges points sideways, height = 0 keeps waits exact, and alpha makes them see-through.
outlier.shape = NA hides duplicate marks and removes nobody from the data; the jitter layer still draws the patients with the longest waits. The alpha setting also makes the dots see-through, not solid.
geom_boxplot() draws medians, quartiles and whiskers, not mean bars. The jitter layer changes only where points are drawn, with height = 0 leaving each waiting time exactly where it is.
The alpha = 0.2 belongs to the jitter layer, so it makes the points see-through, not the boxes. The second line's argument concerns the boxplot's own outlier points.
Answer: the second line draws the boxes without their own outlier points, and the third draws every patient once as a see-through dot nudged sideways. The information needed is what each geom and argument adds to the picture, read one line at a time as in Table 6.5. geom_boxplot() draws the median, quartiles and whiskers for each clinic, and outlier.shape = NA only stops it drawing its own points beyond the whiskers, because geom_jitter() already draws every patient; in that layer width = 0.2 nudges each point sideways, height = 0 leaves each waiting time exactly where it is, and alpha = 0.2 sets the opacity to 20%, so crowded waits look darker. The option about removing patients is tempting, but outlier.shape = NA hides duplicate marks and removes nobody from the data; the jitter layer still shows the patients with the longest waits. (Section 3, 5.4.3 Reading a Figure's Code in Plain Words, Table 6.5, and 5.5.2 A Numeric Variable Across Groups; Objectives 5.4 and 5.5)

2. A colleague computed a small data frame, rates, with one row per health region and a column pct_vacc that holds the percentage of adults vaccinated against influenza. The code ggplot(rates, aes(x = region)) + geom_bar() draws a bar of height 1 for every region. Why, and what is the fix?

The problem is not the scale of the values. geom_bar() ignores pct_vacc and counts rows, so dividing the percentages would still give a bar of height 1 for every region.
region is a categorical variable on the x axis, which is what geom_bar() expects. The bars have height 1 because geom_bar() counts the single row for each region.
A histogram bins one numeric variable along a number line, so it would not draw one bar per region. The values are already computed, so the bars should take their heights from the data.
geom_bar() counts the rows in each category, which suits one row per person; when the heights are already in the data, geom_col() with the value mapped to y draws them.
Answer: geom_bar() counts the rows in each category, so the fix is to map pct_vacc to y and use geom_col(). The information needed is the shape of the data frame (one row per region, with the values already computed) and what each geom does. geom_bar() needs only x and counts the rows in each category, which suits a file with one row per person; here each region appears in one row, so every count is 1. geom_col() draws bars whose heights are supplied in the data, as in the bar chart of mean loneliness by age group built from aggregate(). The histogram option is tempting because the percentages are numeric, but a histogram bins one numeric variable along a number line and would not draw one bar per region. (Section 3, 5.6 Counting Rows or Drawing Values: geom_bar() and geom_col(), Figure 6.13; Objective 5.6)

3. A scatterplot of self-reported daily steps against age for 1,900 adults has jittered, see-through points and a curve from geom_smooth(method = "loess"). The curve is flat at about 7,500 steps from age 20 to 50, falls to about 5,000 steps at age 75, and turns upward after 85. The grey band is narrow between ages 25 and 65 and very wide above 80, where 14 of the 1,900 adults are. Which statement is correct?

The grey band shows uncertainty about where the average lies at each age, not the spread of individual adults. Most adults lie outside it, so being outside the band does not make a point an outlier.
The loess band is a 95% confidence band for the average at each age, and it widens where data are few; the upturn after 85 rests on very few adults and deserves the least trust.
A straight line would not reveal the true relationship; it would hide the levelling and the fall that the curve shows. A loess smoother is used precisely because it can bend.
The upturn rests on very few adults, where the band is widest, so it is not reliable evidence. A narrow band describes certainty about the average, not that every adult walks the same amount.
Answer: the band is the 95% confidence band for the average, and the upturn after 85, where data are few, deserves the least trust. The information needed is what the curve and the band represent and how many people lie behind each part of the curve. A loess smoother summarizes the average of y across the values of x and can bend, and the grey band shows the uncertainty about where that average lies; it is not the spread of individual people, most of whom lie outside it. The band widens where data are few, so the upturn after 85, which rests on a handful of the 14 adults over 80, is the part to trust least. The first option is the most tempting misreading: a band for the average is much narrower than the spread of individual step counts. A straight line would hide the levelling and the fall that the curve shows. (Section 3, 5.7 Reading a Scatterplot with a Smoother, Figure 6.15; Objective 5.7)

4. A figure shows body mass index against age with a straight line in each panel of facet_wrap(~ region). The table gives the number of adults in each panel. In region F the line slopes downward, while the lines in the other five panels slope upward. What should the briefing say about region F?

RegionABCDEF
Adults in the panel410385352298419
A slope estimated from 9 people is very uncertain, with a wide band, so the downward line could easily be chance. It should not be reported as a finding, let alone the main one.
An exploratory figure shows small groups rather than hiding them. The panel is kept and its size stated so that readers do not over-interpret it.
A line fitted to very few people is highly uncertain, so the panel is shown with each panel's size stated and no conclusion is drawn about that region.
Changing the display adds no information about region F. Six overlapping colours would be hard to read, and the 9 points would be hidden among the other regions' points.
Answer: the panel holds only 9 adults, so it is shown with its size stated and not over-interpreted. The information needed is the number of adults behind each panel: regions A to D have about 300 to 400 each, but region F has 9, so a slope estimated from 9 people is very uncertain and its confidence band will be wide. An exploratory figure shows small groups rather than hiding them, and the caption or text states the panel sizes (as Table 6.6 does for the non-binary panels of Figure 6.17) so that readers do not over-interpret them. The single-panel option is tempting, but six overlapping colours are hard to read and the 9 points would be hidden among about 1,500 others; changing the display adds no information about region F. (Section 3, 5.8 Comparing Groups with Facets, Table 6.6; Objective 5.8)

5. A figure for a printed report shows systolic blood pressure against age, with one line for women (red) and one for men (green). Some readers have red-green colour-vision deficiency, and the report will be printed in black and white. Which change lets every reader tell the two lines apart?

The Okabe-Ito palette stays distinguishable with the common forms of colour-vision deficiency, and a second cue that does not depend on colour, such as line type, survives black-and-white printing.
Thicker lines do not help readers who cannot tell red from green, and a black-and-white print removes the colour difference altogether. A safe palette and a non-colour cue are needed.
A theme changes the background, grid lines and text, not the colours mapped to the data. theme_minimal() leaves the red and green lines as they are.
The dpi in ggsave() controls the sharpness of the file, not which colours readers can separate. Red and green stay indistinguishable for some readers and vanish in greyscale.
Answer: use colour-blind-safe Okabe-Ito colours and add line type as a second cue. The information needed is the two problems: about 8% of men of northern European ancestry cannot tell red from green, and a black-and-white copy removes colour altogether. The Okabe-Ito palette was designed to stay distinguishable with the common forms of colour-vision deficiency, and a second cue that does not depend on colour (line type, point shape or separate panels) survives greyscale printing. The theme option is tempting, but a theme changes only the background, grid lines and text, not the colours mapped to the data, and the dpi set in ggsave() controls the sharpness of the file, not which colours readers can separate. (Section 3, 5.9 Colour-Blind-Safe Palettes, Themes and Saving a Figure; Objective 5.9)

6. A health unit surveyed 1,350 adults in five regions about recreational screen time and sleep. Four regions hold between 290 and 360 respondents each, and the North region holds 40. The figure was drawn with the code below. In the four large regions, the curves for both sexes fall from about 8.0 hours of sleep at 1 hour of screen time to about 6.5 hours at 6 hours, and the grey bands widen beyond 6 hours, where only 30 respondents are found. In the North panel the two curves cross. Which reading of the code and the figure is correct?

ggplot(svy, aes(x = screen_hrs, y = sleep_hrs, colour = sex)) +
  geom_jitter(width = 0.2, height = 0.2, alpha = 0.2) +
  geom_smooth(method = "loess") +
  facet_wrap(~ region) +
  labs(x = "Recreational screen time (hours per day)",
       y = "Sleep (hours per night)", colour = "Sex") +
  theme_minimal()
The grey bands show uncertainty about the average sleep at each screen time, not the spread of individuals. The crossing in the North panel rests on only 40 people and could easily be chance.
colour = sex inside aes() maps sex to colour, giving each sex its own colour and curve, and facet_wrap() draws one panel per region. A cross-sectional association also does not show that screen time shortens sleep.
Small panels are shown with their size stated, not removed. Switching to a straight line would hide the shape of the relationship that the loess curves can show.
colour = sex gives each sex its own curve, facet_wrap() gives each region a panel, and the bands show where averages are uncertain; the curves describe an association, least trustworthy where data are few.
Answer: each sex has its own colour and curve in one panel per region, the curves describe an association, and they deserve the least trust where data are few. The information needed is what each line of code adds to the picture, read as in Table 6.5, and how many people sit behind each part of the figure. colour = sex maps sex to colour, so each sex gets its own colour, legend entry and curve; geom_jitter() spreads out people who gave the same answers; geom_smooth(method = "loess") draws a curve of average sleep that can bend, with a grey 95% confidence band for that average; and facet_wrap(~ region) draws one panel per region on the same axes. In the large regions, adults with more screen time tend to sleep less, which describes the data and does not show that screen time shortens sleep. The bands widen where few people are, beyond 6 hours of screen time, and the two curves in the North panel rest on only 40 people between them, so the crossing could easily be chance; the panel is shown, its size is stated, and no conclusion is drawn from it. The first option is the common misreading of the band, which shows uncertainty about the average, not the spread of individual people. (Section 3, 5.4.3 Reading a Figure's Code in Plain Words, 5.7 Reading a Scatterplot with a Smoother and 5.8 Comparing Groups with Facets; Objectives 5.4, 5.7 and 5.8)
● Complete the quiz to continue.
Section 4 of 4

Many Variables and Combined Figures

⏱ Estimated time: 50 minutes
Lesson 6 · Section 4

Many Variables and Combined Figures

An overview of many variables, and one figure that brings the briefing together.

Correlation

Pearson's r for every pair of variables

Loneliness withrDirection
Depressive symptoms (PHQ-2)0.47Positive
Anxiety symptoms (GAD-2)0.43Positive
Social support−0.40Negative
Life satisfaction−0.37Negative
Age0.07Close to zero

The strongest correlation in the matrix is between depression and anxiety (r = 0.59).

Heatmap

A correlation matrix drawn as colours

Correlation heatmap of six variables, red for positive and blue for negative correlations, with the value printed on each square.
Pearson correlations among six briefing variables (n = 3,083).
Interpretation

What a correlation heatmap cannot show

Cause

A correlation fits several causal stories, including reverse causation and common causes.

Curves

Pearson's r measures straight-line association, so a curved pattern can give r near zero.

Shape

One number per pair hides outliers, clusters and the number of people involved.

Marginal histograms

A relationship and two distributions in one display

Jittered scatterplot of social support against loneliness with a red downward line, a histogram of support along the top with spikes at whole numbers, and a histogram of loneliness along the right side.
Drawn with ggMarginal(p_scatter, type = "histogram").
Combining panels

The briefing as one figure

Four-panel figure: A histogram of loneliness, B boxplots by age group, C scatterplot of support and loneliness with a red line, D correlation heatmap.
(p_hist + p_box) / (p_scatter + p_heat) + plot_annotation(tag_levels = "A")
Publication figures

Conventions for a manuscript figure

  • Each axis has a label in words, with units or the range of the scale.
  • Plain labels replace R variable names such as life_sat.
  • Panels carry letters, and legends say what colours and marks mean.
  • Colours remain readable with colour-vision deficiency and in greyscale.
  • Text is legible at the printed size, and the file is saved at about 300 dpi.
Captions

What a figure caption states

ElementExample from the briefing figure
What the figure showsLoneliness and social connection
Data source and yearCanadian Social Connection Survey, 2021
People included3,083 respondents with complete data
Measures and rangesUCLA 3-item scale (3 to 9); support (1 to 7)
What the marks meanBoxes, whiskers, jitter, line and 95% band
Lesson summary

From a first look to a finished figure

StepTool
Look before modellingPlots of every variable and relationship
Choose the displayThe question and the variable types
Check the data quicklyBase R: hist(), boxplot(), plot(), pairs()
Build the figureggplot2: data, aesthetics, geoms, facets, themes
Combine and captionpatchwork, ggsave() and a caption that stands alone

Overview

The working group's third question asks how loneliness goes together with social support and mental health, which involves several variables at once. One scatterplot per pair would answer it slowly, because six variables have fifteen pairs. This section shows how to summarize many pairs in one display with a correlation matrix and a heatmap, when Spearman's rank correlation is the better summary, and what such a summary cannot show. It then adds marginal histograms to a scatterplot, combines the briefing plots into one multi-panel figure checked against the conventions of a publication figure, and writes the caption that lets the figure stand on its own. The finished figure is the kind of exploratory figure that a research paper based on survey data often includes.

Learning Objectives

  • 6.1 Compute a correlation matrix with cor(), display it as a heatmap in ggplot2, and describe a correlation in words by its direction and strength.
  • 6.2 Explain when Spearman's rank correlation is preferred to Pearson's r.
  • 6.3 Explain why a correlation heatmap shows association and cannot identify causes, curved relationships or the shape of the data.
  • 6.4 Build and read a scatterplot with marginal histograms using ggExtra::ggMarginal().
  • 7.1 Combine several ggplot2 plots into one labelled figure with the patchwork package, save it with ggsave(), and check it against the conventions of a publication figure.
  • 7.2 Write a caption that states what the figure shows, so that it stands on its own, and keep interpretation for the results text.

6.1 Correlation Matrices and Heatmaps

6.1.1 The Correlation Matrix

The Pearson correlation coefficient, written r, measures how closely two numeric variables follow a straight line. By this we mean that it describes how tightly the points of a scatterplot cluster around a straight line, and whether that line slopes up or down. It ranges from −1 (all points on a line sloping downward) through 0 (no straight-line relationship) to +1 (all points on a line sloping upward). Its sign gives the direction: a positive r means that people with higher values on one variable tend to have higher values on the other, and a negative r means that higher values on one tend to go with lower values on the other. Its size, ignoring the sign, gives the strength: the closer r is to −1 or +1, the more tightly the points cluster around a line, and the closer it is to 0, the more loosely they are scattered. Cohen (1988) suggested that values of about 0.1, 0.3 and 0.5 be called small, medium and large in the behavioural sciences, and these labels are a rough guide only, since what counts as a strong association depends on the field and the question.

The function cor(), given a data frame of numeric columns, returns a correlation matrix with one row and one column per variable. Each cell holds the correlation between its row variable and its column variable. The diagonal holds the correlation of each variable with itself, which is always 1, and the matrix is symmetric, so the values above the diagonal repeat those below it. Six variables give 36 cells and 15 distinct correlations.

Table 6.7 gives the correlation matrix for the six numeric briefing variables, as cor() prints it in Activity 6.4.

Table 6.7. Pearson correlations among the six numeric variables in the briefing file (n = 3,083).

lonelinesssupportdepressionanxietylife_satage
loneliness1.00−0.400.470.43−0.370.07
support−0.401.00−0.38−0.300.35−0.04
depression0.47−0.381.000.59−0.41−0.07
anxiety0.43−0.300.591.00−0.33−0.06
life_sat−0.370.35−0.41−0.331.000.00
age0.07−0.04−0.07−0.060.001.00

In the briefing file, loneliness has moderate positive correlations with depressive symptoms (0.47) and anxiety symptoms (0.43), and moderate negative correlations with social support (−0.40) and life satisfaction (−0.37). Its correlation with age is close to zero (0.07). The strongest correlation in the matrix is between depressive and anxiety symptoms (0.59), two measures that often move together. To find the strongest correlation with one variable, read along its row, skip the 1 on the diagonal, and compare the sizes whatever their sign.

A correlation is put into words about people, not units. The value of −0.40 for social support and loneliness means that people with more social support tend to report less loneliness, a moderate negative association. It has no units and it is not a slope. This is because r is calculated after each variable has been put on a common standardized scale, so it measures how consistently the two variables move together, not how much one changes when the other changes. It therefore does not mean that loneliness falls by 0.40 points for each extra point of support (that would be a regression slope, which depends on the units of both variables), and it is not a percentage, either of the variation in loneliness or of the people in the file. One practical point also applies to cor(): it returns NA for any pair with a missing value unless told how to handle them, for example with use = "complete.obs". The briefing file has no missing values, so this does not arise here.

A matrix of 36 numbers is slow to read, and the same matrix can be drawn in colour.

6.1.2 Drawing the Matrix as a Heatmap

A correlation heatmap draws each cell of the matrix as a coloured square, so that the pattern can be seen at a glance. Drawing it in ggplot2 takes two steps. The matrix is first turned into a long data frame with one row per pair. By this we mean that the square table, in which each correlation sits where a row meets a column, is rearranged into a list in which each correlation has its own row, with one column naming the first variable, one naming the second and one holding the value. ggplot2 needs the data in this form because each aesthetic is mapped to a column: a square can be placed at its horizontal and vertical position and coloured by its value only if those three pieces of information are stored as three columns. In the code, as.table() treats the matrix as a table, and as.data.frame() turns that table into three columns, here renamed var1, var2 and r. Then geom_tile() draws one square per row, with var1 mapped to x, var2 to y and r to fill, and geom_text() prints the value on each square. Figure 6.19 shows the heatmap for the six briefing variables.

Correlation heatmap of loneliness, support, depression, anxiety, life satisfaction and age. The diagonal is dark red (1). Loneliness, depression and anxiety are pink to red with each other (0.43 to 0.59) and blue with support and life satisfaction (minus 0.30 to minus 0.41). Age is nearly white in every square.
Figure 6.19. Pearson correlations among six briefing variables (n = 3,083). Red squares are positive and blue squares negative, with white at zero, and each square shows its value.

The colour scale is a diverging scale, set with scale_fill_gradient2(): blue for negative values, white at zero and red for positive values, with limits = c(-1, 1) so that the colours keep the same meaning in any heatmap. A diverging scale is one that runs outward in two directions from a neutral middle colour. It suits correlations because zero is a meaningful midpoint: it separates negative associations from positive ones. As a result, the hue of a square shows the direction of the correlation at a glance, and the depth of the colour shows its strength, with pale squares near zero and deep colours near −1 or +1. Blue and red remain distinguishable for people with red-green colour-vision deficiency, which a red-to-green scale would not.

The heatmap shows a clear structure. Loneliness, depression and anxiety form a block of positive correlations, and each of the three is negatively correlated with social support and life satisfaction, which are positively correlated with each other (0.35). Age is close to white in every square. For the briefing, this panel gives the working group an overview of the third question in one picture.

Every square in the heatmap is a Pearson correlation, which is not always the best single number for the kinds of scores a survey produces.

6.2 When Spearman's Rank Correlation Is Preferred

Pearson's r assumes numeric variables and is sensitive to outliers and to skew, because it is calculated from the actual values, so a few very large or very small values can pull it up or down. For ordinal scores or skewed variables, Spearman's rank correlation, computed with cor(..., method = "spearman"), uses the ranks of the values instead. By this we mean that each variable is first replaced by the positions of its values when they are sorted from smallest to largest (1 for the smallest, 2 for the next, and so on, with tied values sharing the average of their positions), and the correlation is then calculated on those ranks. Because a rank records only order, an extreme value simply receives the highest or lowest rank, however far it lies from the others, so Spearman's correlation is less affected by extreme values. In addition, it measures whether one variable tends to rise as the other rises, whether or not the rise follows a straight line, which suits ordered ratings whose categories are not evenly spaced. For loneliness and social support, Spearman's correlation is −0.36, close to the Pearson value of −0.40. When the two values are similar, the Pearson value is not being driven by a few extreme values; when they differ noticeably, the shape of the data (skew, a pile-up at a top-coded value, or a relationship that rises and then levels off) deserves a scatterplot before either number is reported.

The cue for Spearman's correlation is usually in the description of the variable: an ordered rating, a strongly skewed count, or a long tail of extreme or piled-up values. Activity 6.4 prints both coefficients for loneliness and social support, so the comparison can be checked on your own output. Whichever coefficient is used, it remains one number per pair, and one number per pair has limits.

6.3 What a Correlation Heatmap Cannot Show

A correlation heatmap shows the strength and direction of straight-line association between each pair of variables. Three common readings go beyond what it shows: that a correlation identifies a cause, that a correlation near zero means there is no relationship, and that one number describes the data. Expand each panel of the accordion below to see why the heatmap cannot support each reading, with examples from the briefing.

A correlation does not identify a cause

The correlation of 0.47 between loneliness and depressive symptoms is consistent with several causal stories. Loneliness might increase depressive symptoms. Depression might lead people to withdraw from others and feel lonelier, which is reverse causation. Other factors, such as poor physical health, bereavement or low income, might increase both, which is confounding by a common cause. All three stories, and combinations of them, would produce the same square on the heatmap. The CSCS 2021 data were collected at one time point, so the order of events cannot be established from them. Separating these explanations requires study design, causal diagrams of the kind drawn in earlier courses, and models that adjust for confounders, and even then a cross-sectional survey supports causal claims only weakly. A heatmap is therefore a map of where to look, and the briefing should describe it as showing which measures go together.

A correlation near zero does not mean no relationship

Pearson's r measures how well a straight line describes the data. The correlation between age and loneliness is 0.07, yet the loess curve in Section 3 showed a clear pattern: loneliness dips around age 30, rises into the fifties and falls after 70. Rises and falls in different parts of the age range cancel out in a single straight-line summary. By this we mean that a straight line fitted across the whole age range must choose one overall direction, and where loneliness rises with age in some parts of the range and falls in others, those stretches pull the line in opposite directions and leave it almost flat. A near-white square means only that no straight line fits well, and a scatterplot is needed to see whether a curve does.

A correlation hides the shape and size of the data

Anscombe's quartet in Section 1 showed four datasets with the same correlation of 0.82 and completely different shapes, including a curve, an outlier and a relationship created by a single point. A heatmap reduces each pair to one number, so it hides outliers, clusters, floors and ceilings, and it says nothing about how many people contributed. Strong or surprising squares should be followed up with a scatterplot of that pair.

Each panel ends at the same place: a heatmap shows association only, and it points to the pairs of variables that deserve a closer look. One way to take that closer look is a scatterplot of one pair with the distribution of each variable along its margins.

6.4 Scatterplots with Marginal Histograms

A scatterplot shows how two variables go together, and a histogram shows how one variable is distributed. A scatterplot with marginal histograms shows both at once: the scatterplot sits in the middle, with a histogram of the horizontal variable along the top and a histogram of the vertical variable along the right side. The function ggMarginal() from the ggExtra package adds the margins to a finished ggplot2 scatterplot. Its argument type can be "histogram", "density" or "boxplot". Figure 6.20 shows loneliness against social support with a histogram of each variable along its margin.

Scatterplot of social support from 1 to 7 against loneliness from 3 to 9, with jittered grey points and a red line sloping down from about 8 to about 4.3. Along the top, a histogram of support is skewed to the left, with most values between 4 and 6 and tall spikes at 4, 5 and 7. Along the right, a histogram of loneliness has separate bars at each whole score.
Figure 6.20. Loneliness score against social support with marginal histograms (n = 3,083). Points are jittered vertically; the red line is a least-squares line with its 95% confidence band.

The marginal histogram of social support shows a distribution skewed to the left: most people have scores between 4 and 6, and a long, thin tail runs toward low support. It also shows spikes at whole-number scores. In the briefing file, 170 people have a score of exactly 5, 136 exactly 4 and 117 exactly 7. The support score is the mean of twelve items, so a whole-number score is what a person who gives the same answer to every item receives, and a score of 7 requires the highest answer on all twelve. Such answers can reflect genuine views or a respondent moving quickly through the questions (sometimes called straight-lining), and the plot is a reason to look more closely before analyzing the score. The marginal histogram of loneliness shows the seven separate values of that score.

The object that ggMarginal() returns is a finished drawing, and it is not a ggplot2 object, so further layers cannot be added to it and patchwork cannot place it in a grid without extra steps. It can be saved with ggsave(). The combined briefing figure (Figure 6.21) therefore uses the scatterplot without its margins.

The marginal histograms add what the heatmap could not: the shape of each variable, here the spikes in the support score at whole-number values. With the displays for many variables in hand, the briefing plots can now be assembled into a single figure.

7.1 Combining Panels into a Publication Figure

7.1.1 Combining Panels with patchwork

A multi-panel figure lets a reader see several related displays together, with consistent styling and one caption. The patchwork package combines ggplot2 plots with arithmetic-like operators. A plus sign (+) or a vertical bar (|) places plots side by side, a forward slash (/) stacks them, and parentheses group them. The expression (p_hist + p_box) / (p_scatter + p_heat) therefore gives two plots in the top row and two in the bottom row. The function plot_annotation() adds a title, a subtitle or a caption to the whole figure, and its argument tag_levels = "A" labels the panels A, B, C and D in reading order, which the caption can then refer to. The function plot_layout() controls the number of columns, the relative widths and heights of panels, and whether legends are collected in one place. Figure 6.21 shows the draft briefing figure that this expression produces from the histogram, the boxplots, the scatterplot and the heatmap.

A four-panel figure titled Loneliness and social connection, CSCS 2021, n = 3,083. Panel A: histogram of loneliness scores. Panel B: boxplots of loneliness by age group. Panel C: jittered scatterplot of social support and loneliness with a red line. Panel D: correlation heatmap of six variables.
Figure 6.21. The draft briefing figure produced in Activity 6.4 with patchwork. The caption for this figure is given in Worked Example 6.1 later in this section.

The figure is saved with ggsave("briefing_figure.png", figure, width = 11, height = 8.5, dpi = 300). A four-panel figure needs a larger size than a single plot, so that each panel keeps legible text, and 11 by 8.5 inches suits a full page turned sideways or a slide. When the figure is placed in a narrower space, such as one column of a journal, it is better to save it at that width and adjust the text size with base_size than to shrink a large image.

The draft in Figure 6.21 still shows R variable names in panel D, which is one of the things a check against the conventions of a publication figure catches.

7.1.2 Conventions of a Publication Figure

An exploratory figure in a manuscript has a different audience from the analyst's quick looks. Readers see it once, often without the text nearby, and they should be able to understand it without access to the code. Tufte (1983) argued that a graphic should spend its ink on the data and remove decoration that carries no information, which he called chartjunk. Table 6.8 lists the conventions most journals expect and checks the draft briefing figure against them; read down the third column to see which conventions the draft already meets and which it does not.

Table 6.8. Conventions of a publication figure, checked against the draft briefing figure (Figure 6.21).

ConventionWhat it meansDraft briefing figure
Axis labels in words, with units or ranges"Loneliness score (3 to 9)" in place of lonelinessMet in panels A to C
Plain labels for variables"Life satisfaction" in place of life_satNot yet met in panel D, which shows R names
Panel lettersA, B, C and D, referred to in the captionMet with tag_levels = "A"
Legends that name their scaleThe fill legend in panel D is labelled rMet, though "Pearson r" would be clearer
Readable coloursColour-blind-safe palettes that also work in greyscaleMet (teal, a red line, a blue-white-red scale)
Legible text at printed sizeText of about 8 points or more when printedMet at 11 by 8.5 inches
ResolutionAbout 300 dots per inch for printMet with dpi = 300
Sample sizeThe number of people shown, in the title or captionMet in the title (n = 3,083)
No chartjunkNo three-dimensional effects, shadows or decorationMet; a plain theme would lighten it further

The draft meets most conventions in Table 6.8 and falls short on the plain labels in panel D, which Worked Example 6.1 fixes. Even a figure that meets every convention still needs a caption.

7.2 Writing a Figure Caption

7.2.1 What a Caption States and What It Leaves to the Text

A caption makes a figure stand on its own. Most journals place the caption below the figure, beginning with the figure number and a short title, followed by sentences that tell the reader what is shown. A good caption states what the figure displays and for whom: the data source and year, the number of people included and the reason others were excluded, the measures and their ranges, and the meaning of every panel and mark, such as what the boxes, whiskers, lines and bands represent and whether points were jittered. The caption describes the figure; the interpretation, such as which differences matter, belongs in the results text that refers to it.

Draft captions usually fall short in the same few ways, so a draft can be checked against them: a causal claim (such as "loneliness causes depression"), which a caption should not make and a single survey cannot support; R variable names in place of plain measures with their ranges; no data source, year or number of people, and no statement of who was excluded; and panels and marks that are not explained, such as what the boxes, points, lines and bands show. A caption that fixes all four lets the figure stand on its own.

Worked Example 6.1 applies these rules to the briefing. It makes the changes that Table 6.8 called for in the draft in Figure 6.21, gives the full caption of the finished figure, and shows the three sentences of text that accompany the figure in the briefing. As you read it, check the caption against each element listed above, and notice that every interpretation is kept out of the caption and placed in the text.

Worked Example 6.1: The finished briefing figure and its caption

The analyst replaces the R names in panel D with plain labels (loneliness, social support, depression, anxiety, life satisfaction and age), applies theme_minimal() to all four panels, and saves the figure at 11 by 8.5 inches and 300 dpi. The caption reads as follows.

Figure 1. Loneliness and social connection among respondents to the 2021 wave of the Canadian Social Connection Survey (n = 3,083 respondents with complete data on all variables shown; 962 of 4,045 respondents were excluded for missing data). (A) Distribution of scores on the three-item UCLA Loneliness Scale (range 3 to 9; higher scores indicate more loneliness). (B) Loneliness score by age group; boxes show the median and interquartile range, whiskers extend to the most extreme values within 1.5 times the interquartile range, and points show values beyond the whiskers. (C) Loneliness score against perceived social support (Multidimensional Scale of Perceived Social Support, range 1 to 7); points are jittered vertically by up to 0.2 and drawn with transparency, and the line is a least-squares line with its 95% confidence band. (D) Pearson correlations among loneliness, social support, depressive symptoms (PHQ-2, 0 to 6), anxiety symptoms (GAD-2, 0 to 6), life satisfaction (1 to 10) and age.

In the one-page briefing, three sentences of text accompany the figure. Loneliness scores were spread across the whole scale, with the most common scores at 5 and 6 and a group of respondents at the maximum score. Median loneliness was one point higher among respondents aged 45 and older than among younger respondents, although scores varied widely within every age group. Loneliness was moderately correlated with lower social support, more depressive and anxiety symptoms and lower life satisfaction; these are associations in a survey collected at one time and do not show which factors cause which.

The caption describes and the text interprets, so together they complete the briefing without either one doing the other's work. The same division of labour holds when a figure of this kind appears in a research paper.

7.2.2 The Figure Within a Manuscript

An exploratory figure is one part of the results of a research paper. STROBE, the reporting guideline for cohort, case-control and cross-sectional studies (von Elm et al., 2007), lists what the results section should report. Item 13 asks for the number of people at each stage of the study and the reasons they were not included, which the caption of the briefing figure gives (3,083 of 4,045 respondents, with 962 excluded for missing data). Item 14 asks for the characteristics of the participants and the number with missing data for each variable, usually in a Table 1, and item 15 asks for summaries of the outcome, such as the distribution in panel A. Item 16 asks for the main results: unadjusted and adjusted estimates with their confidence intervals, and a statement of which confounders were adjusted for and why. These estimates are usually set out in a results table with one column for the crude estimates and one for the adjusted estimates, for example from a logistic model.

The figure and the tables then support the rest of the paper. The abstract reports the main estimate with its confidence interval in one or two sentences. The discussion interprets the results, compares them with earlier studies and states the limitations; for the CSCS, these include the cross-sectional design, which limits causal claims, and the online recruitment, which limits generalization to the Canadian population.

Activity 6.4 practises the whole section in R. It computes the correlation matrix in Table 6.7 and Spearman's correlation, draws the heatmap in Figure 6.19 and the scatterplot with marginal histograms in Figure 6.20, and assembles and saves the four-panel briefing figure in Figure 6.21. Download the code and run it along with the text version below; the questions ask you to interpret the matrix, to answer two claims that go beyond what a correlation shows, and to check the figure against Table 6.8 and write its caption, which you can compare with Worked Example 6.1.

R Activity 6.4: many variables and a combined figure

Text version of the activity: the code of the run-along script, its console output and notes on reading it.

brief <- read.csv("https://heal-learn.ca/r-activities/data/cscs_brief.csv")   # course data, read from the website
brief$age_group <- factor(brief$age_group,
  levels = c("16 to 29", "30 to 44", "45 to 64", "65 and older"))   # order of the groups
nrow(brief)          # people with complete data in the 2021 wave
# install.packages(c("ggplot2", "ggExtra", "patchwork"))   # once, if R cannot find them
library(ggplot2)
library(ggExtra)     # ggMarginal(): histograms in the margins of a plot
library(patchwork)   # + and / combine ggplot2 plots into one figure
Console output
[1] 3083

The briefing file holds the 3,083 respondents to the 2021 wave of the Canadian Social Connection Survey with no missing value on the briefing variables (line 33 of Activity 6.2 printed that 962 of the 4,045 respondents were left out). read.csv() reads text columns as plain text, so line 8 restores the order of the age groups.

vars <- c("loneliness", "support", "depression", "anxiety", "life_sat", "age")
cor_mat <- round(cor(brief[, vars]), 2)     # Pearson correlations, rounded
cor_mat
round(cor(brief$loneliness, brief$support, method = "spearman"), 2)
Console output
loneliness support depression anxiety life_sat age loneliness 1.00 -0.40 0.47 0.43 -0.37 0.07 support -0.40 1.00 -0.38 -0.30 0.35 -0.04 depression 0.47 -0.38 1.00 0.59 -0.41 -0.07 anxiety 0.43 -0.30 0.59 1.00 -0.33 -0.06 life_sat -0.37 0.35 -0.41 -0.33 1.00 0.00 age 0.07 -0.04 -0.07 -0.06 0.00 1.00 [1] -0.36

Reading along the loneliness row and skipping the 1.00 on the diagonal, the strongest correlation is with depressive symptoms (0.47), then anxiety symptoms (0.43) and social support (−0.40); strength is judged by the size of r whatever its sign. The two halves of the matrix mirror each other, so six variables give 15 distinct correlations. Spearman's rank correlation of loneliness with support is −0.36, close to the Pearson value of −0.40.

cor_long <- as.data.frame(as.table(cor_mat))   # one row for each pair
names(cor_long) <- c("var1", "var2", "r")
head(cor_long)
p_heat <- ggplot(cor_long, aes(x = var1, y = var2, fill = r)) +
  geom_tile(colour = "white") +              # one coloured square per pair
  geom_text(aes(label = r), size = 3) +      # the value on each square
  scale_fill_gradient2(low = "#2166AC", mid = "white", high = "#B2182B",
                       limits = c(-1, 1)) +  # blue negative, red positive
  labs(x = NULL, y = NULL, fill = "r")
p_heat
Console output
var1 var2 r 1 loneliness loneliness 1.00 2 support loneliness -0.40 3 depression loneliness 0.47 4 anxiety loneliness 0.43 5 life_sat loneliness -0.37 6 age loneliness 0.07
Correlation heatmap of loneliness, support, depression, anxiety, life satisfaction and age: red squares for positive and blue squares for negative correlations, white at zero, with the value printed on each square. Loneliness, depression and anxiety are red with each other and blue with support and life satisfaction; age is nearly white everywhere.

as.table() treats the matrix as a table and as.data.frame() turns it into a long data frame with one row per pair (36 rows), because geom_tile() draws one square per row, with var1 mapped to x, var2 to y and r to fill. scale_fill_gradient2() sets a diverging scale with white at zero, and limits = c(-1, 1) fixes its ends at the possible range of r.

set.seed(2021)
p_scatter <- ggplot(brief, aes(x = support, y = loneliness)) +
  geom_jitter(height = 0.2, width = 0, alpha = 0.15) +
  geom_smooth(method = "lm", colour = "#CC0033") +
  labs(x = "Social support (1 to 7)", y = "Loneliness score (3 to 9)")
ggMarginal(p_scatter, type = "histogram")    # a histogram on each axis
head(sort(table(brief$support), decreasing = TRUE), 3)   # the tallest spikes
Console output
`geom_smooth()` using formula = 'y ~ x' `geom_smooth()` using formula = 'y ~ x' 5 4 7 170 136 117
Jittered scatterplot of social support (1 to 7) against loneliness score (3 to 9) with a red line sloping down from about 8 to about 4.3. Along the top, a histogram of support is skewed to the left, with most values between 4 and 6 and tall bars at 4, 5 and 7. Along the right, a histogram of loneliness has separate bars at each whole score.

The message appears twice because ggMarginal() builds the scatterplot twice while it adds the margins; it is not an error. The histogram of support along the top is skewed to the left, with most people between 4 and 6 and a long, thin tail toward low support, and it has spikes at whole-number scores: 170 people score exactly 5, 136 exactly 4 and 117 exactly 7. A whole-number score is what a person who gives the same answer to all twelve support items receives. ggMarginal() returns a finished drawing, not a ggplot2 object, so patchwork cannot place it in a grid without extra steps, and the combined figure uses p_scatter without its margins.

p_hist <- ggplot(brief, aes(x = loneliness)) +
  geom_histogram(binwidth = 1, fill = "#0B7B6B", colour = "white") +
  labs(x = "Loneliness score (3 to 9)", y = "Number of people")
p_box <- ggplot(brief, aes(x = age_group, y = loneliness)) +
  geom_boxplot(fill = "#E6F3F0") +
  labs(x = "Age group", y = "Loneliness score (3 to 9)")
figure <- (p_hist + p_box) / (p_scatter + p_heat) +   # + side by side, / stacked
  plot_annotation(tag_levels = "A",                     # label the panels A to D
    title = "Loneliness and social connection, CSCS 2021 (n = 3,083)")
figure
ggsave("briefing_figure.png", figure, width = 11, height = 8.5, dpi = 300)
Console output
`geom_smooth()` using formula = 'y ~ x' `geom_smooth()` using formula = 'y ~ x'
Four-panel figure titled Loneliness and social connection, CSCS 2021 (n = 3,083). Panel A: histogram of loneliness scores. Panel B: boxplots of loneliness by age group. Panel C: jittered scatterplot of social support and loneliness with a red line. Panel D: correlation heatmap of six variables, whose labels show R names such as life_sat.

The plus sign places two plots side by side, the parentheses group each pair, and the forward slash stacks the two rows, giving a two-by-two grid. plot_annotation(tag_levels = "A") labels the panels A to D in reading order and adds the title. Typing figure draws the figure and ggsave() draws it again to save it, so the message from the scatterplot panel appears twice. The file is 11 by 8.5 inches at 300 dots per inch, or 3,300 by 2,550 pixels.

Files for this activity Data Code Answer key

R Reflect on what you just ran

Answer each question from your own output, as on the final exam: read the matrix, say what a correlation can and cannot show, and check and caption the figure.

1. From cor_mat, which variable is most strongly related to loneliness, and in which direction? Interpret that correlation in one sentence. Social support is skewed: why is the Spearman correlation of loneliness with support (−0.36) useful here, and what does it show beside the Pearson value?

Model answerDepressive symptoms are most strongly related to loneliness (r = 0.47), followed by anxiety symptoms (0.43) and social support (−0.40); strength is judged by the size of r whatever its sign. Respondents with more depressive symptoms tend to report more loneliness, a moderate positive association, although many individuals depart from that trend. Spearman's correlation uses the ranks of the values, so it is less affected by skew and by a few extreme values, and it suits scores built from rating items such as these. Because it is close to the Pearson value (−0.36 against −0.40), the Pearson value is not being driven by a few extreme values.

2. A member of the working group reads the heatmap and writes: “Social support reduces loneliness (r = −0.40), and age is unrelated to loneliness (r = 0.07).” Respond to both claims, and say what you would look at next.

Model answerThe correlation of −0.40 shows only that people with more social support tend to report less loneliness, a moderate negative association in a survey collected at one time. It cannot show which comes first or rule out common causes: loneliness could lower the support people perceive (reverse causation), or poor health could both reduce support and raise loneliness (confounding), so the claim should be reworded as an association. A correlation of 0.07 rules out only a straight-line relationship; the loess curve in Section 3 showed loneliness dipping around age 30, rising into the fifties and falling after 70, and these rises and falls cancel out in r. Before saying anything about age, I would look at a scatterplot of loneliness against age with a loess smoother, and I would follow up any strong or surprising square with a scatterplot of that pair.

3. Run your own code: at the end of your script, type figure & theme_minimal() and run it. What does & do, and which convention of a publication figure does the figure still not meet? Then write a caption for the figure that would let a reader understand it without reading the text.

Model answerThe & applies theme_minimal() to all four panels, giving them one plain theme (a plus sign would change only the last panel); panel D still shows R names such as life_sat, which should be replaced with plain labels such as “Life satisfaction”. Caption: Loneliness and social connection among respondents to the 2021 wave of the Canadian Social Connection Survey (n = 3,083 with complete data on all variables shown; 962 of 4,045 respondents were excluded for missing data). (A) Distribution of the three-item UCLA Loneliness Scale score (range 3 to 9; higher scores mean more loneliness). (B) Loneliness score by age group; boxes show the median and interquartile range, whiskers extend to the most extreme values within 1.5 times the interquartile range, and points show values beyond the whiskers. (C) Loneliness score against perceived social support (range 1 to 7); points are jittered vertically and drawn with transparency, and the line is a least-squares line with its 95% confidence band. (D) Pearson correlations among loneliness, social support, depressive symptoms (PHQ-2, 0 to 6), anxiety symptoms (GAD-2, 0 to 6), life satisfaction (1 to 10) and age, from blue (negative) through white (zero) to red (positive). The caption describes the figure and makes no causal claim; which differences matter belongs in the results text.
Saved.

Your output reproduces Table 6.7 and Figures 6.19 to 6.21, so each step of the section, from a single correlation to a captioned figure, has now been carried out on the briefing data.

This section has summarized many variables with a correlation matrix and a heatmap, set out the limits of that summary, added marginal histograms to a scatterplot, combined the briefing plots into one figure, and described the conventions of a publication figure and its caption. The key takeaways below summarize these ideas, the knowledge check reviews them, and the final page of the lesson brings the four sections together before the final assessment.

Key Takeaways

  • 6.1 A correlation matrix, produced by cor() on a set of numeric columns, gives Pearson's r for every pair of variables. Its diagonal is always 1, the values above the diagonal repeat those below it, and k variables give k(k − 1)/2 distinct correlations (15 for the six briefing variables). A heatmap draws each cell as a coloured square on a diverging scale (blue for negative, white at zero and red for positive, with limits = c(-1, 1)), after the matrix has been reshaped into one row per pair and drawn with geom_tile(). To interpret a correlation, give its direction from the sign and its strength from its size, using Cohen's rough guide of about 0.1, 0.3 and 0.5 for small, medium and large, and put it into words about people: r = −0.40 means that people with more social support tend to report less loneliness, a moderate negative association. When asked for the strongest correlation with a variable, read along its row, skip the 1 on the diagonal, and compare the sizes whatever their sign.
  • 6.2 Pearson's r measures straight-line association between numeric variables, and it can be pulled about by outliers and skew. Spearman's rank correlation, from cor(..., method = "spearman"), works on the ranks of the values instead, so it is preferred for ordinal scores, skewed variables, and variables with extreme or piled-up values. Comparing the two is a useful check: for loneliness and social support, Spearman's value is −0.36 and Pearson's is −0.40, so the Pearson value is not being driven by a few extreme values. When the two differ noticeably, look at a scatterplot before reporting either number. In an exam question, the cue for Spearman is usually in the description of the variable: an ordered rating, a strongly skewed count, or a long tail of extreme values.
  • 6.3 A correlation heatmap shows the direction and strength of straight-line association between pairs of variables, and three common readings go beyond that. It cannot show a cause: the correlation of 0.47 between loneliness and depressive symptoms fits loneliness raising depressive symptoms, depression leading people to withdraw and feel lonelier (reverse causation), or a common cause such as poor physical health raising both (confounding), and a survey collected at one time cannot establish the order of events. A correlation near zero does not mean no relationship: age and loneliness have r = 0.07, yet the loess curve in Section 3 showed loneliness dipping around age 30, rising into the fifties and falling after 70, and these rises and falls cancel out in a single straight-line summary. And one number hides the shape of the data, including outliers, clusters, floors and ceilings and how many people contributed, as Anscombe's quartet showed. So when someone says a correlation shows that one variable reduces another, or that two variables are unrelated, reply that the correlation shows association only, name the other explanation (reverse causation, confounding or a curve), and ask for a scatterplot of that pair.
  • 6.4 A scatterplot with marginal histograms shows a relationship and both distributions at once: the scatterplot sits in the middle, with a histogram of the horizontal variable along the top and a histogram of the vertical variable along the right side. In R, you build an ordinary ggplot2 scatterplot and pass it to ggMarginal(p_scatter, type = "histogram") from the ggExtra package ("density" and "boxplot" are the other types). Read it in two steps: describe the relationship in the middle, then describe each margin for skew, spikes or piled-up values. In Figure 6.20, the margin showed that social support is skewed to the left and has spikes at whole-number scores (170 people at exactly 5), which is the score a person receives by giving the same answer to all twelve items, and a reason to look more closely before analyzing the score. Note that ggMarginal() returns a finished drawing, not a ggplot2 object, so you cannot add layers to it or place it in a patchwork grid without extra steps.
  • 7.1 The patchwork package combines finished ggplot2 plots into one figure with simple operators: + or | places plots side by side, / stacks them, and parentheses group them, so (p_hist + p_box) / (p_scatter + p_heat) gives two plots on top and two below. plot_annotation(tag_levels = "A") labels the panels A, B, C and D in reading order so that the caption can refer to them, and the figure is saved with ggsave() at a size that keeps every panel's text legible (11 by 8.5 inches at 300 dpi for the briefing). Before calling a figure finished, check it against the conventions in Table 6.8: axis labels in words with units or ranges, plain names rather than R names such as life_sat, panel letters, legends that name their scale, colour-blind-safe colours, legible text, about 300 dpi, the sample size, and no chartjunk. When asked to critique a combined figure, work through that list, name each convention that is not met, and say how you would fix it.
  • 7.2 A caption is what lets a figure stand on its own, so that a reader can understand it without the text. It states what the figure shows and for whom: the data source and year, the number of people included and why others were excluded, the measures with their ranges, and what every panel and mark means (boxes, whiskers, points, lines and bands, and whether the points were jittered). It describes rather than interprets: which differences matter belongs in the results text, and a survey collected at one time cannot support a causal claim such as "loneliness causes depression". When you write or check a caption, test it against the four common failings (a causal claim; R names in place of plain measures; no source, year or number of people; and panels or marks that are not explained), and use the caption in Worked Example 6.1 as your model.
Knowledge check: this section

1. An analyst prints the Pearson correlations among five variables from a community health survey of 1,500 adults: daily steps, body mass index, systolic blood pressure, a stress score and age. Which statement correctly describes the strongest correlation with the stress score?

> round(cor(dat[, c("steps", "bmi", "sbp", "stress", "age")]), 2)
       steps   bmi   sbp stress   age
steps   1.00 -0.31 -0.18  -0.42 -0.25
bmi    -0.31  1.00  0.36   0.12  0.15
sbp    -0.18  0.36  1.00   0.09  0.48
stress -0.42  0.12  0.09   1.00 -0.05
age    -0.25  0.15  0.48  -0.05  1.00
Strength is judged by the size of r whatever its sign, so a negative correlation can be the strongest. A BMI correlation of 0.12 is weak compared with the correlation for steps.
The 0.48 is the correlation of blood pressure with age, not with stress. Only the stress row or column answers the question.
The variable and value are right, but r is not a slope. It has no units and does not give the change in stress per step; it describes the strength and direction of a straight-line association.
Reading the stress row and judging strength by the size of r, steps is strongest; the negative sign means more steps go with less stress, and a value near 0.4 is moderate on Cohen's rough guide.
Answer: daily steps, r = −0.42, a moderate negative association. The information needed is the stress row (or column) of the matrix, ignoring the 1.00 on the diagonal: steps −0.42, BMI 0.12, blood pressure 0.09 and age −0.05. Strength is judged by the size of r whatever its sign, so −0.42 is the strongest; the negative sign gives the direction (more steps go with less stress), and a value near 0.4 is moderate on Cohen's rough guide of about 0.1, 0.3 and 0.5 for small, medium and large. The blood pressure option is tempting, but 0.48 is the correlation of blood pressure with age, not with stress, and reading r as the change in stress per step treats it as a slope, which it is not. (Section 4, 6.1.1 The Correlation Matrix, Table 6.7; Objective 6.1)

2. A colleague reads a correlation heatmap from the same survey, which was collected at one time, and writes two sentences: "Age is unrelated to stress (r = −0.05). Walking more lowers stress (r = −0.42), so a walking program will reduce stress." Which response is correct?

A correlation summarizes only the strength and direction of a straight-line association. It says nothing about cause and can miss a curved relationship entirely.
An r near zero rules out only a straight-line relationship, since a curve can give r close to zero, and a cross-sectional correlation fits reverse causation and confounding as well as the colleague's story.
The first sentence repeats the near-zero mistake, since a curve can give r close to zero. The second fails because of the design, not the size of r: a one-time survey cannot show cause.
No size of correlation shows a causal effect. Reverse causation and confounding would produce the same value, so the design, not the size of r, limits the conclusion.
Answer: both sentences go beyond what a correlation can show. The information needed is what Pearson's r measures: the strength and direction of straight-line association between two variables measured at one time. A curved pattern, such as stress that is higher in young and in old adults than in middle age, can give r close to zero because the rises and falls cancel out, so a scatterplot with a smoother is needed before saying the variables are unrelated. The correlation of −0.42 fits several causal stories: walking reduces stress, stress reduces walking (reverse causation), or a common cause such as poor health lowers walking and raises stress (confounding). The option that accepts the first sentence is tempting, but it repeats the near-zero mistake, and the second sentence fails because of the design, not the size of r. (Section 4, 6.3 What a Correlation Heatmap Cannot Show; Objective 6.3)

3. A scatterplot of systolic blood pressure against the number of alcoholic drinks in the past week has marginal histograms added with ggMarginal(p, type = "histogram"). The histogram along the top shows drinks piled up near zero with a long right tail and about 15 adults above 40 drinks; the histogram on the right shows blood pressure as roughly symmetric. The Pearson correlation is 0.16 and the Spearman correlation is 0.06. Which interpretation is correct?

The margins are histograms of each variable on its own; the relationship is in the scatterplot. Averaging Pearson and Spearman values hides the information that their gap provides.
A lower value is not a less accurate one. For a skewed variable the rank-based Spearman value is often the better summary, because Pearson's r is sensitive to extreme values.
The marginal histogram shows the skew in drinks, and because Spearman uses ranks while Pearson is sensitive to extreme values, a noticeable gap suggests a few heavy drinkers drive the Pearson value.
Top-coding appears as a spike at the largest value, not as a long tail. The adults above 40 drinks are real answers, examined in the scatterplot rather than set to missing.
Answer: drinks are strongly right-skewed, and the lower rank-based Spearman value suggests that a few extreme values drive the Pearson value. The information needed is what each part of the display shows and how the two correlations differ. The middle of the display is the scatterplot (the relationship), and the margins are histograms of each variable on its own, which here show a long right tail in drinks. Pearson's r is sensitive to extreme values and skew, whereas Spearman's rank correlation uses only the order of the values; when the two are similar the Pearson value is not being driven by a few points, but 0.16 against 0.06 suggests the 15 heavy drinkers carry much of it, so the scatterplot is examined before either number is reported. The option calling Spearman less accurate is tempting, but a lower value is not a less accurate one; for a skewed variable the rank-based value is often the better summary. The tail is not top-coding, which would appear as a spike at the largest value. (Section 4, 6.2 When Spearman's Rank Correlation Is Preferred and 6.4 Scatterplots with Marginal Histograms; Objectives 6.2 and 6.4)

4. A colleague combined four plots with (p_a + p_b) / (p_c + p_d) + plot_annotation(tag_levels = "A") and saved the result with ggsave("draft.png", figure, width = 11, height = 8.5, dpi = 72). In the draft, the axes of panel D show the R names sbp and steps_day, and panel C colours women red and men green. Which assessment against the conventions of a publication figure is correct?

Against the publication conventions, R names, a low print resolution and red-green colours fall short, while tag_levels = "A" gives the panel letters that a caption can refer to.
R names fall short of the plain-label convention, since readers of a figure do not have the script, and 72 dpi is far below the resolution expected for print.
Panel letters are a convention of publication figures, letting the caption refer to each panel. About 300 dpi is expected for print, so 72 dpi falls short.
In patchwork, + places plots side by side and / stacks rows, so the parentheses and slash give a two-by-two grid, not a single row.
Answer: the R names, the 72 dpi resolution and the red-green colours fall short, and the panel letters meet the convention. The information needed is the checklist in Table 6.8: axis labels in words with units, plain labels for variables, panel letters that the caption can refer to, named legends, colour-blind-safe colours, legible text, about 300 dots per inch for print, the sample size and no chartjunk. R names fail the plain-label convention, 72 dpi is far below the 300 dpi expected for print, and red against green fails readers with red-green colour-vision deficiency, while tag_levels = "A" labels the panels A to D in reading order, as the convention requires. The layout option misreads patchwork: + places two plots side by side, the parentheses group each pair, and / stacks the two rows, giving a two-by-two grid. (Section 4, 7.1.1 Combining Panels with patchwork and 7.1.2 Conventions of a Publication Figure, Table 6.8; Objective 7.1)

5. A two-panel figure from a 2024 community health survey shows (A) a histogram of self-reported daily steps and (B) boxplots of systolic blood pressure for three age groups, with each adult drawn as a jittered point. Of the 1,900 adults surveyed, 1,640 had complete data. Which caption lets the figure stand on its own?

This caption makes a causal claim and uses R names. A caption describes the figure without interpreting it and gives the source, the sample, the measures and the meaning of the marks.
It names both panels but omits the source, the sample size and the meaning of the marks, and it interprets the figure and recommends a program, which belongs in the results and discussion.
It gives the sample size but omits the source, year, exclusions, measures and the meaning of the marks, and comments on the analyst's effort, which a caption does not include.
A caption that stands on its own gives the data source and year, the number included and excluded, the measures with units, and the meaning of every panel and mark, without interpreting the figure.
Answer: the last caption, which gives the source, the sample, the measures and the meaning of every mark. The information needed is the list of caption elements: the data source and year, the number of people included and how many were excluded (1,900 − 1,640 = 260), the measures with their units or ranges, and the meaning of every panel and mark, such as the boxes, whiskers and jittered points. That caption states all of these and describes the figure without interpreting it. The caption that names both panels is tempting, but it omits the source, the sample size and the marks, and it interprets the figure and recommends a program, which belongs in the results and discussion; the first caption makes a causal claim and uses R names. (Section 4, 7.2.1 What a Caption States and What It Leaves to the Text, Worked Example 6.1; Objective 7.2)

6. A survey of 1,120 staff in long-term care homes recorded weekly overtime hours, usual sleep (hours per night), a burnout score (0 to 30, higher means more burnout), days of sick leave in the past year (most staff took 0 to 3 days, and a few took more than 40) and age. Which summary of the output below is correct?

> round(cor(staff[, vars]), 2)
          overtime sleep burnout sick_days   age
overtime      1.00 -0.31    0.44      0.18 -0.12
sleep        -0.31  1.00   -0.52     -0.15  0.05
burnout       0.44 -0.52    1.00      0.26 -0.03
sick_days     0.18 -0.15    0.26      1.00  0.09
age          -0.12  0.05   -0.03      0.09  1.00
> round(cor(staff$burnout, staff$sick_days, method = "spearman"), 2)
[1] 0.14
A negative correlation can be the strongest, and sleep's correlation is larger in size than overtime's. A lower Spearman value is not less accurate; for skewed sick days it may be the better summary.
r is not a slope, so it does not give the change in burnout per hour of sleep. A correlation near zero rules out only a straight-line relationship, not every relationship with age.
Strength is judged by the size of r whatever its sign, so sleep is strongest, and the sign gives the direction. A noticeable Pearson and Spearman gap for a skewed variable calls for a plot.
A correlation from a survey collected at one time does not show cause, whatever its size. Averaging the two correlations would also hide what their gap reveals about the skewed sick days.
Answer: sleep is most strongly related to burnout, and the gap between the two correlations for sick days calls for a closer look at the data. The information needed is the burnout row of the matrix and what each correlation measures. Reading along the burnout row and skipping the 1.00 on the diagonal, the values are 0.44 (overtime), −0.52 (sleep), 0.26 (sick days) and −0.03 (age); strength is judged by the size of r whatever its sign, so sleep is the strongest, and the negative sign means that staff who sleep less tend to report more burnout. Sick days are strongly skewed, Pearson's r is sensitive to skew and to a few extreme values, and Spearman's correlation uses the ranks of the values, so when the two differ noticeably the shape of the data deserves a plot, such as a scatterplot with marginal histograms, before either number is reported. The overtime option is tempting, but a negative correlation can be the strongest one. Reading r as the change in burnout per hour of sleep treats it as a slope, which it is not; a correlation near zero rules out only a straight-line relationship; and a correlation from a survey collected at one time does not show a cause. (Section 4, 6.1.1 The Correlation Matrix, 6.2 When Spearman's Rank Correlation Is Preferred, 6.3 What a Correlation Heatmap Cannot Show and 6.4 Scatterplots with Marginal Histograms; Objectives 6.1, 6.2 and 6.4)
● Complete the quiz to continue.
Final Assessment

Lesson 6: Final Assessment

16 questions • 100% required to pass

Bringing It All Together

This lesson followed one request from start to finish: a one-page visual briefing on social connection for a regional loneliness working group, drawn from the 2021 wave of the Canadian Social Connection Survey. Section 1 set out why the analyst looks before modelling. Anscombe's quartet showed four datasets with the same means, standard deviations, correlation and fitted line but completely different patterns, and a decision guide matched the display to the number and types of the variables in a question. A gallery of poor displays showed what a truncated axis, a bar chart of means, a crowded pie chart and an overplotted scatterplot conceal.

Section 2 prepared the briefing file in base R (3,083 people with complete data on nine variables) and used quick base R plots to check it. A histogram revealed a spike of 116 people at the top value of household size, where larger totals had been recorded as 20, and heaping at 40 hours in reported hours of work, problems that the numeric summaries did not show. Section 3 rebuilt the plots with ggplot2, using the layered grammar of data, aesthetics, geoms, scales, facets and themes. A loess smoother showed that loneliness dips around age 30 and rises to a second high point in the fifties, facets compared the relationship between social support and loneliness across gender and age groups, and the Okabe-Ito palette kept the colours readable. Section 4 summarized many variables with a correlation heatmap, explained why a heatmap shows association and cannot show cause, curves or shape, added marginal histograms to a scatterplot, and combined four panels into one figure with a caption.

The same habits apply to every analysis: each variable is plotted before it is summarized, each relationship is plotted before it is modelled, and each figure carries labels and a caption that let it stand on its own. One question remains open: whether the scale scores plotted here, such as the loneliness and social support scores, measure what they are meant to measure, and how reliably they do so.

Key Takeaways from this lesson

  1. You can explain why exploratory analysis comes first: it uses plots and simple summaries to see the shape of each variable, any unusual values and the relationships between variables before a model is fitted, whereas confirmatory analysis tests a question set in advance. Anscombe's quartet shows why this matters, because four very different datasets share the same means, standard deviations, correlation and fitted line. Whenever an exam question gives you only summary numbers, say what a plot could reveal that the numbers cannot.
  2. You can choose a display by first naming how many variables a question involves and what type each one is: a histogram or a bar chart for one variable; a scatterplot, boxplots with points or a bar chart of percentages for two; and a correlation heatmap, a scatterplot matrix or facets for many. You can also decide between colour, for two or three groups that overlap little, and separate panels, for many, overlapping or unequal groups. The most useful reminder is to identify the variable types before naming the display, and to compare percentages within groups when the groups differ in size.
  3. You can spot a truncated bar axis, bars of means without spread, a pie chart with many slices, overplotting, a dual axis and colours that rely on telling red from green, explain what each one hides, and propose a better display. In a critique question, give each problem, what a reader could get wrong because of it, and its fix, and check the labels, the colours and the context (sample size and data source) as well as the type of display.
  4. You can read a histogram for skew, ceilings, heaping, top-coding, impossible values and missing-value codes, and a boxplot for centre, spread, skew and points beyond the whiskers. For every data problem you can respond by checking the codebook, deciding how to handle the problem, and reporting the decision with the number of people affected. Remember that points beyond the whiskers are flagged, not wrong, and that a right-skewed count with a variance well above its mean points toward a count model.
  5. You can read base R or ggplot2 code line by line and say in plain words what each part does for the figure: which variables are mapped to the axes, colour or fill; what the boxes, the jittered see-through points and the smoothed lines with their confidence bands show; and what facets, palettes, themes and saving with code add. You can then describe what the figure shows, including how many people sit behind each panel. The most useful reminder is that the grey band shows uncertainty about the average, not the spread of people, and that small panels are shown and their size stated, but they are not over-interpreted.
  6. You can read a correlation matrix, a correlation heatmap and a scatterplot with marginal histograms, describe a correlation by its direction and strength in plain words about people, and say when Spearman's rank correlation is preferred to Pearson's r. You can also explain why a correlation cannot show a cause, a curved relationship or the shape of the data. When someone draws a causal or "unrelated" conclusion from a correlation, name the other explanation and ask for a scatterplot of the pair.
  7. You can check a combined multi-panel figure against the conventions of a publication figure (plain labels with units, panel letters, named legends, colour-blind-safe colours, legible text, about 300 dpi, the sample size and no chartjunk) and write a caption that lets it stand on its own. The caption states the data source and year, the number of people and who was excluded, the measures and their ranges, and what every panel and mark means, and it describes rather than interprets, leaving any conclusion to the results text.

The final assessment covers all four sections. All 16 questions must be answered correctly (100%) to finish the lesson.

Final Knowledge Assessment

Final assessment: the 16 questions

1. A colleague reports that the correlation between minutes of daily physical activity and resting heart rate is −0.80, gives the fitted straight line, and says that no plot is needed because "the numbers say it all". Which reply is best?

Anscombe's quartet refutes this: a strong correlation and the same fitted line arose from a curve, an outlier and one influential point as well as a straight cloud.
A correlation of any size is consistent with very different patterns, so a plot is needed whether r is near zero or large.
Very different patterns can share the same correlation and fitted line, so only a scatterplot shows whether a straight line fairly describes the data.
The four sets in Anscombe's quartet also share their means and standard deviations, so these do not describe the data fully either.
Answer: a scatterplot is needed, because Anscombe's quartet shows that very different patterns can share the same correlation and fitted line. The information needed is what a summary statistic can and cannot show. In the quartet, four datasets share the same means, standard deviations, correlation (0.82) and fitted line, yet only one of them is an even cloud around a straight line; the others are a curve, a line pulled by one outlier and a relationship created by one point. A correlation of any size, including −0.80, is therefore consistent with very different patterns, and only the plot shows which one is present. The first option is the conclusion that the quartet was built to refute, and the last fails for the same reason, since the four sets also share their means and standard deviations. (Section 1, 1.2 Why Summary Statistics Alone Can Mislead; Objective 1.2)

2. A briefing must compare the distribution of life satisfaction (whole numbers from 1 to 10) across four household income groups, which hold between 220 and 1,150 respondents. Which display suits this best?

A numeric variable across the groups of a categorical one calls for side-by-side boxplots, and showing each respondent as a point adds the spread and the very different group sizes.
A bar chart of means shows four numbers and hides the spread, shape and size of each group, which the comparison of distributions needs.
Pie charts compare angles, which people judge less accurately than positions on a common scale, and four separate pies make the groups hard to compare.
A single histogram mixes the four groups, so no difference between them can be seen. The question asks for a comparison across groups.
Answer: side-by-side boxplots with the points shown. The information needed is the type of each variable: life satisfaction is numeric and income group is categorical with four groups, so the request is a numeric variable across groups, which Figure 6.2 matches to side-by-side boxplots. Showing each respondent as a (jittered) point adds the spread and the number of people in each group, which here differ fivefold. The bar chart of means is tempting, but it hides the spread, shape and size of each group, and a single histogram mixes the groups so that no difference between them can be seen. (Section 1, 2.1 Matching the Display to the Question, Figure 6.2; Objective 2.1)

3. A survey asks whether respondents are up to date with routine vaccinations. The analyst prints the counts and the percentages within each area below. A colleague's bar chart of counts shows a much taller "not up to date" bar for urban respondents. Which statement is correct?

> table(dat$area, dat$vaccines)                       # counts

        Not up to date Up to date
  Rural             45        105
  Urban            270       1080
> round(100 * prop.table(table(dat$area, dat$vaccines), margin = 1), 1)   # percentages within each area

        Not up to date Up to date
  Rural             30         70
  Urban             20         80
Comparing counts between groups of very different sizes mainly shows that the urban group is larger. How often each group is not up to date is read from percentages within each area.
This is the wrong comparison: it describes where the not-up-to-date respondents live, which mostly reflects how few rural respondents there are, not how often each group is up to date.
Groups of different sizes can be compared, using percentages within each group. That is exactly what the second table provides.
Two categorical variables with groups of different sizes are compared with percentages within each group, and those show rural respondents more often not up to date despite their smaller count.
Answer: 20% of urban and 30% of rural respondents are not up to date, which a bar chart of percentages within each area shows. The information needed is the second table, which R prints as the percentage within each area (each row adds to 100), because two categorical variables with groups of different sizes are compared with percentages within each group. Rural respondents are more often not up to date (30% against 20%) even though their count is smaller, and the bar chart of counts mainly shows that the urban group is much larger (1,350 against 150 respondents). The option about 45 of the 315 is the wrong-comparison mistake: it describes where the not-up-to-date respondents live, which mostly reflects how few rural respondents there are, not how often each group is up to date. (Section 1, 2.1 Matching the Display to the Question, Interactive 6.1, item 5; Objective 2.1)

4. A draft figure plots, for each month, emergency department visits for asthma (red line, left axis from 0 to 400 visits) and the mean concentration of fine particulate matter (blue line, right axis from 10 to 30 micrograms per cubic metre). The two lines cross in June, and a colleague writes that pollution "overtakes" asthma visits in June. Which response is correct?

The crossing depends on the scale ranges the analyst chose, so it carries no information about cause or timing. Changing either axis would move or remove it.
With two vertical axes the crossing depends on the chosen scales, and series in different units cannot overtake each other; two panels sharing the month axis show each series on its own scale.
Starting both axes at zero still leaves two arbitrary scales on one plot, so the position of any crossing remains meaningless.
Pie charts suit a few categories that make up a whole. Visits and pollution are different measures, not parts of a whole, and a monthly series is read best along a time axis.
Answer: the crossing is an artefact of the two scales, and two panels that share the month axis are the better display. The information needed is that the two series are drawn against two vertical scales whose ranges the analyst chose (0 to 400 visits and 10 to 30 micrograms per cubic metre). Changing either range would move the crossing or remove it, so it carries no information, and visits and concentrations cannot "overtake" each other because they are measured in different units. Two panels stacked on a shared time axis let each series be read on its own scale while keeping the months aligned. The zero-axis option is tempting, but starting both axes at zero still leaves two arbitrary scales on one plot. (Section 1, 3.1.1 Four Displays That Conceal the Data, Truncated axis tab, which also covers the dual axis; Objective 3.1)

5. A clinic report shows the main reason for 2,300 walk-in visits as a pie chart with 12 slices, several of them thin, and the manager cannot tell whether skin problems or minor injuries are more common. Which redesign is best?

Labels help, but the three-dimensional effect is chartjunk that distorts the angles further. The problem with many thin slices is that angles are hard to compare.
A histogram needs a numeric axis, and visit reason is categorical. Its touching bars would imply a continuous scale that the reasons do not have.
People judge positions along a common scale more accurately than angles or areas, so a sorted bar chart shows the order and size of every difference among the 12 reasons.
A line chart implies an order between points, but visit reasons have no natural order; alphabetical order would create rises and falls that mean nothing.
Answer: a bar chart sorted by count. The information needed is the variable type (visit reason is categorical, with 12 categories) and how people read displays: they judge positions along a common scale more accurately than angles or areas (Cleveland & McGill, 1984). A sorted bar chart places every reason on one axis, so the order and the size of each difference are clear, whereas a pie chart is acceptable only for two or three categories that make up a whole. The three-dimensional pie is tempting because labels help, but the 3D effect is chartjunk that distorts the angles further; a histogram needs a numeric axis, and a line chart implies an order that the reasons do not have. (Section 1, 3.1.1 Four Displays That Conceal the Data, Pie with many slices tab; Objective 3.1)

6. A histogram of household income (in $1,000s per year) falls smoothly toward higher incomes and then shows a tall bar of 64 households at 250, the largest value. The codebook says that incomes of $250,000 or more were recorded as 250. What is this, and how should it be handled?

A missing-value code stands for no answer, but these households did answer. The codebook says 250 records incomes at or above the limit, so the values are real.
Heaping is rounding to round numbers within the range. A tall bar at the largest value, with the codebook stating incomes above the limit were recorded as the limit, is top-coding.
The value is not impossible; it is the recorded limit for high incomes. Removing these households would drop real high earners and bias the analysis.
A tall bar at the largest value with a codebook rule recording incomes above a limit as the limit is top-coding: the values are real, so they are kept as a top group and reported.
Answer: top-coding, handled by keeping the values as "250 or more", grouping income with a top group, and reporting it. The information needed is the sign on the histogram (a bar at the largest value that is taller than the bars just below it) and the codebook's statement that every income above the limit was recorded as the limit. So 250 means "this much or more": the 64 households are real and their incomes are high but not exact, which is why the value is kept, treated as a top group, labelled as such, and reported with the number of households affected. The missing-value option is tempting because a code is also found in the codebook, but a missing-value code stands for no answer, whereas these households did answer. (Section 2, 4.2.2 A Problem That Cleaning Left Behind: Top-Coding and Heaping, and 4.2.3 Missing-Value Codes and Narrow Bins; Objective 4.2)

7. In a survey of adults who smoke, reported cigarettes per day pile up at 10, 20 and 40, with few answers at 9, 11, 19 or 21. What does this suggest, and how should the analysis treat it?

Pile-ups on round or standard numbers inside the range, with few answers at neighbouring values, are heaping: real but imprecise answers, so small differences are treated with caution or the variable grouped.
Top-coding would show a pile-up only at the largest recorded value. The pile-ups at 10 and 20 point to rounding, and nothing suggests larger values were recorded as 40.
Missing-value codes stand for no answer and usually lie outside the possible range. These are plausible numbers of cigarettes that people rounded, so they are real answers.
Nothing suggests the question limited answers to 40, and the pile-ups at 10 and 20 show rounding to half a pack and a pack. That pattern is heaping, not a ceiling.
Answer: heaping, so small differences are interpreted with caution or the variable is grouped, and the heaping is reported. The information needed is where the pile-ups fall: on round or standard numbers inside the range (10, 20 and 40 are half a pack, one pack and two packs), with few answers at the neighbouring values. That pattern is heaping, also called digit preference; it usually reflects rounding, so the answers are real but imprecise. Heaping cannot be corrected, so the analyst interprets small differences with caution or groups the variable and notes it in the methods. The top-coding option is tempting because 40 is a high value, but the pile-ups at 10 and 20 show rounding, and nothing suggests that larger values were recorded as 40. (Section 2, 4.2.2 A Problem That Cleaning Left Behind: Top-Coding and Heaping, and 4.2.3 Missing-Value Codes and Narrow Bins; Objective 4.2)

8. Boxplots of the number of emergency department visits in the past year, for four income groups, show medians of 2, 1, 1 and 0 (from the lowest to the highest income), long upper whiskers, short lower whiskers and many separate points above the boxes. For all respondents, R prints the output below. What do the plot and these numbers suggest?

> mean(dat$ed_visits)
[1] 1.6
> var(dat$ed_visits)
[1] 6.4
> var(dat$ed_visits) / mean(dat$ed_visits)   # variance divided by the mean
[1] 4
Points beyond the whiskers are flagged by the 1.5 times IQR rule, not shown to be wrong, and large visit counts are possible values. Deleting them would not make linear regression suitable for counts.
Long upper whiskers and points only above the boxes show right skew, and a variance several times the mean is a first sign of overdispersion, so a count model, possibly negative binomial, is suggested.
A variance larger than the mean points to overdispersion in counts, not to normality. The right-skewed boxplots also rule out a normal distribution.
Differing medians make income group a candidate predictor, but no first look guarantees a good fit. The formal checks of a count model still have to be carried out.
Answer: the counts are right-skewed with a variance four times the mean, which suggests a count model and possible overdispersion. The information needed is the shape in the boxplots and the ratio of the variance to the mean. Long upper whiskers, short lower whiskers and points only above the boxes are the signs of right skew, as expected for counts that cannot fall below zero but can be large; the last line of the output shows that the variance is 4 times the mean, and Section 2 (4.3) explains that a variance several times the mean is a first sign of overdispersion, so a negative binomial model may be needed. This first look ignores the predictors, so the formal checks are made on the fitted model, and income group, whose medians differ, is a candidate predictor. The deletion option is tempting, but points beyond the whiskers are flagged by the 1.5 × IQR rule rather than shown to be wrong, and large visit counts are possible values that a count model is designed to handle. (Section 2, 4.3 Reading Boxplots Across Groups; Objective 4.3)

9. A figure was drawn with the code below. The factor smoker has the levels "Does not smoke" and "Smokes". Which plain-words description of colour = sex, geom_smooth(method = "lm") and facet_wrap(~ smoker) is correct?

ggplot(dat, aes(x = age, y = sbp, colour = sex)) +
  geom_jitter(width = 0.3, alpha = 0.2) +
  geom_smooth(method = "lm") +
  facet_wrap(~ smoker) +
  labs(x = "Age (years)", y = "Systolic blood pressure (mmHg)", colour = "Sex") +
  theme_minimal()
A variable inside aes() is mapped, so each sex gets its own colour, not one fixed colour. method = "lm" draws straight lines rather than loess curves, and facets create panels, not a legend.
Mapping colour removes no respondents; it gives each sex its own colour, and the smoother inherits the mapping, drawing one line per sex. facet_wrap() makes panels rather than colours.
Mapping sex to colour inside aes() gives a colour and legend for each sex, the smoother inherits it and draws a straight line with a confidence band per sex, and facet_wrap() repeats the plot for each smoking group.
colour = sex gives colours, not panels; facet_wrap() creates the panels. geom_smooth() fits an average line rather than joining the points, and the panels sit side by side on shared axes.
Answer: each sex has its own colour and straight line with a confidence band, and each smoking group has its own panel. Each part is read as what it adds to the picture, as in Table 6.5 of Section 3. A variable named inside aes() is mapped, so colour = sex gives each sex its own colour and adds a legend, and because the mapping is in ggplot() the smoother inherits it and draws one line per sex; method = "lm" makes each line straight (least squares), with the grey 95% confidence band drawn by default; and facet_wrap(~ smoker) repeats the plot in one panel per level of smoker, sharing both axes so the panels can be compared. The first option is tempting, but it confuses a mapped aesthetic with a fixed setting (a fixed colour is written outside aes()) and confuses "lm" with "loess". (Section 3, 5.4.3 Reading a Figure's Code in Plain Words, Table 6.5, and 5.8 Comparing Groups with Facets; Objective 5.4)

10. A scatterplot shows about 3,000 respondents, with age recorded in whole years and systolic blood pressure recorded to the nearest 2 mmHg. Why does the code draw the points with geom_jitter(width = 0.3, alpha = 0.2) rather than geom_point()?

Jitter does not round values; it adds a small random shift to where each point is drawn so that points sharing a position stop sitting exactly on top of one another.
Jitter moves only the drawn positions; it neither corrects nor changes recorded ages. Its purpose is to separate points that share the same values.
alpha sets the opacity of every point, not the share of points kept. All respondents are still drawn; overlapping see-through points build up into darker areas.
With values recorded on a coarse grid, many respondents share positions and hide one another; jitter separates them slightly and alpha lets overlaps darken, while the data stay unchanged.
Answer: the points are jittered and made see-through so that people who share the same values do not hide one another, and the data are unchanged. The information needed is that both variables take a limited set of values, so with about 3,000 respondents many points fall on exactly the same position (overplotting), and a combination held by fifty people looks the same as one held by a single person. geom_jitter() adds a small random shift to where each point is drawn, and alpha sets the opacity from 0 (invisible) to 1 (solid), so overlapping points build up into darker areas that show where most people are. The option about correcting ages is tempting, but jitter moves only the drawn positions; it neither corrects nor changes the recorded values, and every respondent is still drawn. (Section 3, 5.5.2 A Numeric Variable Across Groups, and Section 2, 5.2 Jitter and Transparency for Overplotted Points; Objectives 5.2 and 5.5)

11. In a scatterplot of systolic blood pressure against age, geom_smooth(method = "lm") draws a straight line with a grey band that is narrow in the middle of the age range and wider at both ends, where there are fewer respondents. What does the band show, and what would geom_smooth(method = "lm", se = FALSE) change?

geom_smooth() estimates the average of y at each x, and its band shows uncertainty about that average, widest where data are few; se = FALSE removes the band and keeps the same line.
The band is for the average, not individuals, and is much narrower than the spread of individual blood pressures. se = FALSE removes only the band; it removes no respondents.
The band describes uncertainty about the average blood pressure at each age, not the spread of age. se = FALSE drops the band; the method argument decides whether the line is straight.
The band reflects uncertainty in estimating the average from the sample, not monitor error. se = FALSE simply stops the band being drawn and leaves the data unchanged.
Answer: the band is the 95% confidence band for the average, and se = FALSE removes the band but keeps the line. The information needed is what geom_smooth() estimates: the average of y at each value of x, here as a straight least-squares line, with a band that shows the uncertainty about where that average lies. The band is not the spread of individual people, most of whom lie outside it, and it widens toward the ends of the age range because fewer respondents are there. The argument se = FALSE keeps the line and drops the band; it changes neither the line nor the data. The option describing the band as the range of 95% of individuals is the most common misreading, since a band for the average is much narrower than the spread of individual values. (Section 3, 5.7 Reading a Scatterplot with a Smoother; Objective 5.7)

12. A figure maps three groups to the default ggplot2 colours, which include a red and a green. Some readers of the report have red-green colour-vision deficiency, and others will print it in black and white. Which plan keeps the three groups distinguishable for all readers?

A theme changes the background, borders and text, not the colours mapped to the data. With theme_bw() the groups would still be told apart only by red and green.
A colour-blind-safe palette such as Okabe-Ito serves readers with colour-vision deficiency, and a second cue that does not rely on colour, such as shapes or panels, survives greyscale printing.
Removing the legend makes the groups impossible to identify for every reader. The fix is a palette and a second cue that all readers can distinguish.
Larger text helps legibility but does nothing to separate red from green or to survive black-and-white printing.
Answer: use a colour-blind-safe palette such as Okabe-Ito and add a second cue (shape or panels). The information needed is the two problems to solve: the default palette includes red and green, which about 8% of men of northern European ancestry cannot tell apart, and a black-and-white print removes colour entirely. The Okabe-Ito palette was designed to stay distinguishable with the common forms of colour-vision deficiency, and a second cue that does not rely on colour, such as point shapes, line types or separate panels, survives greyscale printing. The theme_bw() option is tempting, but a theme changes the background, borders and text, not the colours mapped to the data, so the groups would still be told apart only by red and green. (Section 3, 5.9.1 Palettes and Themes; Objective 5.9)

13. In the briefing file (n = 3,083), the Pearson correlation between social support and loneliness is −0.40. Which sentence is the best interpretation?

r is not a percentage of variation explained. It describes the strength and direction of a straight-line association, read from its sign and size.
r has no units and is not a slope, so it does not give the change in loneliness per point of support. A cross-sectional correlation also does not show what changing support would do.
The negative sign gives the direction, more support with less loneliness, and a size of about 0.4 is moderate on Cohen's rough guide; tend to keeps the sentence descriptive rather than causal.
r is not a proportion of people. It summarizes how the two scores go together across all respondents, not how many respondents fall in any category.
Answer: people with more social support tend to report less loneliness, a moderate negative association. The information needed is the sign and the size of r: the negative sign gives the direction (higher values of one variable go with lower values of the other), and a size of 0.40 is moderate on Cohen's rough guide of about 0.1, 0.3 and 0.5 for small, medium and large. A correlation from one survey describes association, so the sentence says "tend to" and makes no causal claim. The slope option is tempting, but r has no units and is not the change in loneliness per point of support, and the other two read r as a percentage of variation or of people, which it is not. (Section 4, 6.1.1 The Correlation Matrix, Table 6.7; Objective 6.1)

14. In a survey collected at one time, hours spent outdoors each week are correlated with depressive symptoms (r = −0.35), and depressive symptoms are correlated with days of work missed (r = 0.30). A manager concludes: "Time outdoors prevents depression, which prevents missed work, so an outdoor activity program will cut missed work." Which response is correct?

Consistent directions do not establish cause. Each correlation fits reverse causation and confounding as well as the manager's story, and joining two correlations into a chain does not remove that ambiguity.
Spearman's correlation suits skewed variables such as days missed, but changing how association is measured does not turn it into evidence of cause.
Moderate correlations can matter in practice. The conclusion fails because of the design: correlations from one survey show association, not that a program would cut missed work.
A correlation from a one-time survey shows that measures go together; reverse causation and confounding by a common cause would produce the same values, so a causal claim needs a design built to test it.
Answer: the correlations show association only, so the causal chain and the program's effect are not established. The information needed is the design (one survey at one time) and what a correlation measures (association, not cause). Each correlation fits several causal stories, including the manager's, reverse causation (depressive symptoms reduce time outdoors) and confounding by a common cause (poor physical health could lower time outdoors and raise both depressive symptoms and missed work), and all of them would produce the same values; joining two correlations into a chain does not remove that ambiguity. The Spearman option is tempting because Spearman's correlation suits skewed variables such as days missed, but changing how association is measured does not turn it into evidence of cause. (Section 4, 6.3 What a Correlation Heatmap Cannot Show; Objective 6.3)

15. A draft caption reads: "Figure 3. (A) Distribution of loneliness scores (UCLA three-item scale, range 3 to 9). (B) Loneliness score by age group; boxes show the median and interquartile range, whiskers extend to the most extreme values within 1.5 times the interquartile range, and points are jittered." Which addition would most improve it, so that the figure stands on its own?

The draft already explains the measure and marks, so what is missing is who and how many: the data source, the year and the number included and excluded.
Interpretation belongs in the results text, not the caption, and a single cross-sectional survey cannot support a causal claim about ageing.
R code is part of the analysis record, not a caption element. A caption describes the source, the sample, the measures and the marks so that readers can interpret the figure.
Judging importance is interpretation, which belongs in the text. A caption states what the figure shows, not how much it matters.
Answer: add the data source, the year and the number of people included and excluded. The information needed is the list of caption elements: the data source and year, the number of people included and the reason others were excluded, the measures and their ranges, and the meaning of every panel and mark. The draft already names the measure with its range and explains the boxes, whiskers and jittered points, so what is missing is who and how many: the source, the year and the sample (3,083 of 4,045 respondents, with 962 excluded for missing data). The causal sentence is tempting because it says what panel B seems to show, but interpretation belongs in the results text, and a single cross-sectional survey cannot support a causal claim. (Section 4, 7.2.1 What a Caption States and What It Leaves to the Text, Worked Example 6.1; Objective 7.2)

16. A regional food security survey compares psychological distress (Kessler-6 score, 0 to 24, higher means more distress) across four household income groups. A colleague's draft panel is a bar chart of the mean distress score in each income group, with the four bars coloured red, green, orange and brown and the vertical axis titled K6_total. Which critique names the three problems and their fixes?

Bars of means are not the standard display for a numeric variable across groups; they hide the spread, shape and size of each group. Colours that only repeat the labels add nothing and include red and green.
Bars of means conceal spread, shape and group size, so boxplots with jittered points are better; redundant red and green colours are replaced; and an R name gives way to a plain label with the range.
A pie chart suits only two or three categories that make up a whole, and mean scores of separate groups are not parts of a whole. It would also keep the redundant colours.
A theme changes the background, grid lines and text, not the colours mapped to the data, and the resolution does not change what the panel shows. The display, colours and label remain the problems.
Answer: the bars of means, the colours and the axis title all fall short, and each has a fix. The information needed is what bars of means conceal and the smaller faults that a critique also checks: labels, colour and context. Bars of means show four numbers and hide the spread within each income group, the shape of each distribution and the number of people in each group, so boxplots with each respondent drawn as a jittered, see-through point are the better display. The four colours carry no information, because they only repeat the group labels on the axis, and about 8% of men of northern European ancestry cannot tell red from green, so one plain colour, or a colour-blind-safe palette such as Okabe-Ito, is the fix. K6_total is an R variable name with no range, so the axis is labelled in words with its range. The theme_minimal() option is tempting, but a theme changes the background, grid lines and text, not the colours mapped to the data, and the resolution does not change what the panel shows. (Section 1, 3.1.1 Four Displays That Conceal the Data, Means without spread tab, and 3.1.2 Smaller Faults in Draft Figures, and Section 3, 5.9.1 Palettes and Themes; Objective 3.1)

✦ Before submitting: pass every section knowledge check (100%).

🏆 Congratulations!

You have successfully completed this lesson: Exploratory Data Analysis and Visualization.

Your responses have been downloaded automatically.