HSCI 341 · Lesson 2

Surveillance and Sampling

Fundamental Epidemiological Concepts and Approaches

Learning objectives for this lesson:

  1. Explain what public-health surveillance is, using Langmuir's working definition and the idea of closing the loop, and classify a described system as passive, active, sentinel, or syndromic, giving a strength and a limitation of each type.
  2. Trace a notifiable-disease report from the clinician or laboratory to the Public Health Agency of Canada, explain why passive surveillance counts fewer cases than actually occur and why the counted cases are not a random subset, and judge a surveillance source, including the main federal and BC products, on timeliness, completeness, representativeness, sensitivity, and predictive value positive.
  3. Describe the steps of an outbreak investigation (the CDC 10-step framework, the three parts of a case definition, and the roles set out in the Canadian FIORP), and interpret an epidemic curve to name its shape and estimate the likely exposure window from the incubation period.
  4. Calculate attack rates and risk ratios for suspect foods from a line list, identify the most likely vehicle, and use a stratified comparison to separate two foods that were often eaten together.
  5. Distinguish a census from a sample, define the target population, source population, study sample, and sampling frame for a described study, identify a group the frame would miss, and distinguish internal from external validity.
  6. Describe the probability sampling designs (simple random, systematic, stratified, cluster, multistage, and targeted) and the non-probability designs (judgement, convenience, purposive, and chain-referral), name the design used in a described study, and explain how a non-probability sample can bias an estimate and in which direction.
  7. Explain how the standard error, the central limit theorem, sampling weights, and the design effect allow a sample to describe a population with an honest measure of uncertainty.
  8. Distinguish Type I error, Type II error, and statistical power, choose between a sample-size calculation for precision and one for power, and calculate the sample size needed to estimate a proportion or a mean or to compare two groups, with adjustments for clustering, a finite population, and non-response.

This course was developed by Dr. Kiffer G. Card based on Dohoo, I. R., Martin, S. W., & Stryhn, H. (2012). Methods in Epidemiologic Research. VER Inc.

Reference

Glossary: Key Terms, People & Concepts

📚 Reference page, available throughout the lesson

This glossary collects the key concepts, people, and ideas you will meet in this lesson. Use it as a reference while you work through the material, or as a review before assessments. Type in the search box to filter entries.

Surveillance Concepts
Public Health Surveillance Ongoing systematic collection, analysis, and dissemination of health-related data for public health action. Distinguished from research by its operational mandate and rapid feedback loop to action.
Passive Surveillance Surveillance system in which clinicians, labs, or facilities report cases on their own initiative according to legal or professional requirements. Cheap and broad but vulnerable to under-reporting.
Active Surveillance Surveillance staff regularly contact reporters or examine records to actively elicit case reports. More complete and timely than passive systems but resource-intensive; often used during outbreaks.
Sentinel Surveillance High-quality data collected from a selected subset of providers, sites, or populations chosen to represent broader trends. Trades coverage for timeliness and depth.
Syndromic Surveillance Use of pre-diagnostic data (chief complaints, ED visits, school absences, pharmacy sales) to detect outbreaks earlier than confirmed-case reporting allows.
Notifiable (Reportable) Disease A condition that, by law, providers and laboratories must report to public-health authorities (e.g., measles, TB, COVID-19). Lists are maintained provincially in Canada.
Case Definition Standardised criteria (clinical, epidemiologic, laboratory) used to classify suspect, probable, and confirmed cases. Tightening or loosening the definition trades sensitivity against specificity.
Cluster, Outbreak, Epidemic, Pandemic Cluster: aggregation of cases in a place or time. Outbreak: cases above expected (often local). Epidemic: widespread excess (often used interchangeably with outbreak). Pandemic: epidemic across multiple countries or continents.
Index Case (Primary Case) The first case identified in an outbreak investigation (index) or the first case in a transmission chain (primary). They are not always the same person.
Super-Spreader / Super-Spreading Event An individual or event that generates substantially more secondary infections than the population average, typically driven by host, agent, environmental, and behavioural factors converging.
R₀ (Basic Reproduction Number) The expected number of secondary cases produced by a typical infected individual in a fully susceptible population. R₀ > 1 implies epidemic potential (Wikipedia overview).
Rt (Effective Reproduction Number) The average number of secondary cases per infected individual at a specific time, accounting for current immunity and interventions. Rt < 1 indicates a shrinking epidemic.
Attack Rate The proportion of an at-risk population that develops disease during an outbreak (cumulative incidence). Secondary attack rate refers to spread among contacts of a primary case.
Incubation Period Time from exposure to onset of symptoms. Knowing the typical incubation distribution allows back-calculation from onset to likely exposure window.
Generation Time / Serial Interval Generation time: interval between infection of a primary case and infection of a secondary case. Serial interval: interval between symptom onsets in successive cases. Used to estimate Rt.
Herd (Population) Immunity Indirect protection against infection that occurs when a sufficient proportion of the population is immune, reducing the chance that susceptible people contact infectious ones.
Outbreak Investigation Methods
Ten Steps of Outbreak Investigation (CDC) A standardised sequence: prepare, confirm outbreak, verify diagnosis, define and find cases, descriptive epi, hypothesise, test hypotheses, refine hypotheses, implement control, communicate. Steps overlap in practice.
Descriptive Epidemiology (Person, Place, Time) Characterisation of cases by who is affected, where they live or were exposed, and when illness began. The first analytic step in any outbreak investigation.
Epidemic (Epi) Curve A histogram of case onsets over time. Shape suggests transmission pattern: point-source (sharp peak), continuous-source (plateau), propagated/person-to-person (successive peaks at intervals of one incubation period).
Line List A structured table with one row per case capturing demographic, clinical, exposure, and outcome variables. The operational data backbone of outbreak investigation.
Contact Tracing Identification, notification, and follow-up of people exposed to a confirmed case to enable testing, isolation, prophylaxis, or vaccination. Foundational for STI, TB, Ebola, COVID-19 control.
Hypothesis-Generating Interview An open-ended structured interview with a small number of cases (typically 5–10) to elicit possible exposures, such as foods, places, and activities, before a focused case-control or cohort analysis is launched.
Retrospective Cohort (Outbreak) When the at-risk population is well defined (a wedding, a cruise, a daycare), exposed and unexposed cohorts can be reconstructed and attack rates compared directly, the design of choice in defined-population outbreaks.
Case-Control (Outbreak) When the at-risk population is unbounded (community), confirmed cases are compared with appropriate controls on possible exposures to identify the source.
Environmental Sampling & Molecular Subtyping Laboratory analysis of suspect food, water, surfaces, or air, often combined with whole-genome sequencing of patient isolates, to confirm the outbreak source and link cases.
PHAC & BCCDC Public Health Agency of Canada (national surveillance and outbreak coordination) and the British Columbia Centre for Disease Control (provincial counterpart). Both publish public dashboards used in this lesson.
Key People
Alexander Langmuir (1910–1993) Founder of the CDC Epidemic Intelligence Service (EIS) in 1951; established modern shoe-leather outbreak investigation as a core public-health discipline (Langmuir biography).
William Farr (1807–1883) First compiler of statistical abstracts at the General Register Office in London; established systematic mortality surveillance and the foundation of routine vital statistics (Farr biography).
John Snow (1813–1858) London physician whose 1854 investigation of the Broad Street cholera outbreak combined a spot map, water-supply comparison, and shoe-leather case interviews, an enduring template for outbreak epidemiology (Snow biography).
Key Concepts & Ideas
Target Population The full population to which study findings are intended to apply, defined by the research question.
Study (Source) Population The accessible subset of the target population from which a sample can actually be drawn (also called the source population).
Sampling Frame The actual list, register, or operational mechanism used to enumerate and reach members of the study population. Coverage error arises when the frame omits or double-counts target members.
Sampling Unit The element selected at each stage of the sampling process; this could be an individual, household, school, or cluster, depending on design.
Census Data collection from every member of a population. Eliminates sampling error but is usually impractical and may suffer from large non-response.
Probability Sampling Sampling design in which every unit in the frame has a known, non-zero probability of selection. Required for valid statistical inference using survey weights.
Non-Probability Sampling Sampling without known selection probabilities, for example convenience, purposive, snowball, or quota. Useful for hard-to-reach groups but inference to a population is limited.
Sampling Error Random variation in estimates that arises because we observe a sample rather than the whole population. Quantified by the standard error and decreases as sample size increases.
Non-Sampling Error Errors arising from sources other than random sampling, such as coverage, non-response, measurement, and processing. Often more consequential than sampling error.
Type I Error (α) Rejecting a true null hypothesis: a false positive. Conventionally controlled at 0.05.
Type II Error (β) Failing to reject a false null hypothesis: a false negative. Power = 1 − β, conventionally targeted at 0.80 or higher.
Statistical Power The probability that a study correctly detects an effect of a given size when one exists. Depends on sample size, effect size, variability, and α.
Design Effect (DEFF) Ratio of the variance under the actual complex design to the variance under simple random sampling of the same size. DEFF > 1 indicates the effective sample size is smaller than n.
Oversampling Deliberately sampling certain subgroups at higher rates than their population share to improve subgroup precision. Requires weighting in analysis.
Non-Response Bias Systematic error that occurs when individuals who do not respond differ in important ways from those who do.
Sampling Designs & Methods
Simple Random Sample (SRS) A probability sample in which every possible sample of size n from the frame has an equal chance of selection. The reference design against which others are compared.
Systematic Sampling Selecting every kth unit from the frame after a random start. Approximates SRS unless periodicity in the list aligns with the sampling interval.
Stratified Sampling Dividing the population into mutually exclusive strata and sampling independently within each. Improves precision when strata differ in the outcome and ensures representation of subgroups.
Cluster Sampling Sampling intact groups (clusters) such as schools or villages instead of individuals. Reduces field cost but inflates variance because units within a cluster are correlated.
Multistage Sampling A design that selects clusters in successive stages (e.g., regions → neighbourhoods → households → people). Common in national surveys.
Sampling Weight The reciprocal of the selection probability (often adjusted for non-response and post-stratification). Each respondent represents as many people in the population as their weight.
Convenience Sampling Recruiting whoever is most accessible. Fast and cheap but vulnerable to selection bias; common in pilot studies.
Purposive (Judgement) Sampling Researcher selects units believed to be informative or representative based on theoretical criteria. Common in qualitative work.
Snowball Sampling Existing participants refer additional participants. Useful for hidden or stigmatised populations; respondent-driven sampling (RDS) is a variant that partially recovers probability-style inference under strong assumptions.
Quota Sampling Non-probability sampling that fills predefined quotas (e.g., by age and sex) using convenience recruitment within each cell.
No matching entries. Try a different search term.
Section 1 of 5

Surveillance Systems and Canadian Data Sources

⏱ Estimated reading time: 40 minutes
Lesson 2 · HSCI 341

The Phone Call That Drives the Lesson

A duty epidemiologist gets a call about four children with bloody diarrhoea from one school.

Section 1 of 5

Surveillance Systems and Canadian Data Sources

The definition and action loop, the four system types, the Canadian products, and five questions to ask of any source.

Langmuir 1963

What surveillance is, and is not

The ongoing, systematic collection, analysis, interpretation, and dissemination of health-related data for public-health action.Langmuir, 1963; refined by Thacker & Berkelman, 1988

The key distinction: surveillance that does not feed back into decisions, alerts, or programs is bookkeeping, not surveillance.

The loop

The surveillance action loop

Data Sources Data Systems Analysis & Interpretation Action closes the loop
Four types

Passive, active, sentinel, syndromic

Passive

Clinicians and labs report on their own initiative. Cheap and broad; vulnerable to non-random under-reporting.

Active

Public-health staff elicit reports. Higher quality; resource-intensive and narrow.

Sentinel

A designated network reports systematically. Trades coverage for depth and data quality.

Syndromic

Pre-diagnostic signals flag clusters early. High sensitivity; low specificity.

Passive surveillance

Under-reporting is not random

  • Passive systems count only cases that reach care, are tested, and test positive.
  • Severe, urban, and well-served cases are over-represented; mild, rural, and under-served cases are missed.
  • The denominator is the whole population, but the numerator is only the cases that were seen.
Federal layer

Four PHAC headline products

CNDSS

50+ notifiable conditions; annual provincial data; best for long-run trends and inter-provincial comparisons.

FluWatch

Hybrid sentinel + lab + outbreak counts; weekly respiratory-virus picture.

CCDSS

Case definitions on admin records; prevalence and incidence of chronic conditions (diabetes, hypertension, dementia).

CVSD

Statistics Canada mortality spine; life expectancy and excess-mortality analyses.

Provincial layer: BC

BCCDC dashboards worth knowing

  • Respiratory virus data: influenza, RSV, SARS-CoV-2; updated weekly.
  • Communicable Disease Dashboard: reportable diseases, including Salmonella, Campylobacter, STEC, Listeria.
  • Overdose response indicators: death data from the BC Coroners Service.
  • Cancer incidence: published separately by BC Cancer (BC Cancer Registry).
  • STI reports: gonorrhea, chlamydia, syphilis trends.

Back-end platform: Panorama (case and outbreak management, immunization).

Data infrastructure

The long-running layer everything reads from

Vital statistics

Continuous since late 19th century; denominator for life expectancy and mortality surveillance.

Cancer registries

Near-complete case ascertainment; active follow-up; BC Cancer Registry is the provincial source.

Health-admin data

DAD, NACRS, MSP billing. Powers CCDSS and most admin-data studies.

CCHS + wastewater

Survey self-report; wastewater independent of testing behaviour.

Data quality

Five dimensions to interrogate every source

Timeliness
Completeness
Representativeness
Sensitivity
Predictive value positive

Speed vs completeness

Wastewater: days. CNDSS: weeks to months. STIs: large under-count. Cancer registries: near-complete.

Sensitivity vs specificity

Syndromic: catches events early, many false alarms. Passive: misses mild cases, few false alarms.

Carry forward

What to take into the next section

  • Surveillance closes the loop: data count as surveillance only when analysis leads to action.
  • The four system types trade coverage, cost, depth, and speed; most real products are hybrids.
  • Federal products give national trends; provincial products give case-level and regional detail.
  • The five quality dimensions decide which questions a data source can answer.

Overview

This section describes the infrastructure of public-health surveillance, the system that lets a health authority notice that something unusual is happening and respond to it. It begins with what surveillance is and what it means to close the loop, then sorts surveillance systems into four types by who does the work of finding cases. It follows a notifiable-disease report from the clinician to the Public Health Agency of Canada (PHAC) and explains why passive surveillance counts fewer cases than occur, and not a random subset of them. It then surveys the federal and British Columbia surveillance products and the data infrastructure beneath them, and closes with five dimensions on which the quality of any surveillance source can be judged.

Learning Objectives

  • 1.1 State Langmuir's working definition of surveillance and explain what it means to "close the loop."
  • 1.2 Classify a described system as passive, active, sentinel, or syndromic surveillance, and give a strength, a limitation, and a Canadian example of each type.
  • 2.1 Trace the notifiable-disease reporting flow from clinician or laboratory to medical health officer, province, and PHAC, explain why passive surveillance counts fewer cases than occur, and explain why the cases it misses are not a random subset.
  • 2.2 Identify the major federal and BC surveillance products (CNDSS, FluWatch, CCDSS, CVSD, BCCDC dashboards) and the long-running data infrastructure they read from.
  • 2.3 Apply the five dimensions of surveillance data quality (timeliness, completeness, representativeness, sensitivity, and predictive value positive) to any data source.

1.1 Defining Surveillance and Closing the Loop

You are the duty epidemiologist at a regional health authority on a Tuesday afternoon. Three minutes ago, your phone buzzed: a paediatrician at a community clinic just called to say she has seen four children from the same school presenting with bloody diarrhoea over two days. She is requesting stool cultures and wants to know whether you have seen anything similar elsewhere. By the time you put the phone down, you will need to know, quickly, whether four cases is unusual for this organism in this catchment, whether a notifiable disease report has already been filed, who else needs to be looped in, and what data sources you can pull within the next hour. That sequence of questions is what this lesson is about. The infrastructure that lets you answer them is called public-health surveillance, and the structured response that follows is called outbreak investigation, the subject of Section 2.

The classic working definition, attributed to Langmuir (1963) and refined by Thacker & Berkelman (1988), is the ongoing, systematic collection, analysis, interpretation, and dissemination of health-related data for the planning, implementation, and evaluation of public-health practice. Each of those verbs is doing real work. Ongoing distinguishes surveillance from a one-time study. Systematic rules out anecdote. Analysis and interpretation rule out a passive data warehouse. Dissemination for action rules out research that is never returned to the people who can act on it. Langmuir (EIS founder) stressed the regular dissemination of the data and their interpretation to everyone who needs to know, a requirement often summed up as “closing the loop”: information must come back out as decisions, alerts, or programs, otherwise the system is just bookkeeping.

The action that closes the loop can serve any of five purposes, and a published surveillance report is usually doing one or more of them: (1) detecting outbreaks, clusters, and unusual events early; (2) characterizing who is getting sick, where, and why, through descriptive epidemiology by person, place, and time; (3) monitoring trends in incidence, prevalence, and risk factors over time; (4) evaluating the effect of interventions and programs; and (5) planning resource allocation and policy. A single dataset can serve more than one purpose, but the design tradeoffs are different for each. The paediatrician's call in the opening scenario is a test of the first purpose, early detection.

It helps to picture surveillance as a closed loop with four moving parts: data sources (clinical reports, lab results, vital records, administrative data), data systems (the registries, dashboards, and notifiable-disease platforms that ingest and store data), analysis and interpretation (the epidemiologists who turn case counts into rates, anomalies, and narratives), and action (the case finding, control measures, public communications, and policy decisions that flow from the analysis). The loop is closed only when the last stage happens. Each handoff between these stages is also a place where the system can leak: a clinician who never reports a case, a province that does not share data with the federal level, an analyst who does not see a signal in time, or a recommendation that never reaches a decision-maker. When you judge whether a described system counts as surveillance, ask who looks at the data, how often, and what happens when something unusual appears. Ongoing, systematic collection on its own does not close the loop, and neither does passing a file up the reporting chain; the loop closes only when someone acts on the analysis.

Of the four parts of the loop, the data-system layer determines what kinds of signals can be detected at all, and the conventional types of surveillance system differ on a single axis: who does the work of finding cases. Subsection 1.2 sets out those types.

1.2 The Four Types of Surveillance System

Most public-health surveillance systems can be sorted into one of four types: passive, active, sentinel, and syndromic. The syndromic category is the newest, codified by the CDC framework of Buehler and colleagues (2004). The types are not mutually exclusive, since a single disease can be tracked by several at once, but they have different strengths, costs, and biases. Table 2.1 compares the four types by who initiates the report, their strengths and limitations, and a Canadian example of each.

Table 2.1. Passive, active, sentinel, and syndromic surveillance compared by who initiates the report, strengths, limitations, and a Canadian example of each.

TypeWho initiates the reportStrengthsLimitationsCanadian example
Passive Clinicians and labs report when they encounter a notifiable disease. Cheap, broad coverage, mandated by law, runs continuously. Under-reporting (often substantial), variable timeliness, completeness depends on clinician burden. The Canadian Notifiable Disease Surveillance System (CNDSS), aggregated case counts for ~50 reportable conditions submitted by provinces to PHAC.
Active Public-health staff actively contact providers, labs, or households to find cases. Higher case ascertainment, better data quality, useful in outbreak investigations. Resource-intensive, narrow scope, hard to sustain over time. IMPACT (Immunization Monitoring Program ACTive), in which nurse monitors at paediatric hospitals search for vaccine-preventable diseases and adverse events after immunization, and contact tracing during COVID-19 case investigations.
Sentinel A small, designated network of providers reports systematically, trading breadth for depth. High data quality, manageable cost, can collect richer data than passive systems. Not population-representative; trends but not absolute counts. FluWatch's sentinel laboratories (the Respiratory Virus Detection Surveillance System) and its sentinel paediatric hospital network, SPRINT-KIDS (as of October 2026), and the Canadian Paediatric Surveillance Program (CPSP).
Syndromic Real-time signals (chief-complaint codes, EMS calls, OTC drug sales, school absences) flag clusters before lab confirmation. Fast, and can detect events before a definitive diagnosis. Useful for emerging or rare events. Low specificity (lots of false alarms); validation is hard; needs analytic infrastructure. The emergency-department chief-complaint system that KFL&A Public Health began in Kingston, Ontario, in 2004 (Moore, Edgar, & McGuinness, 2008); PHAC's pandemic-era wastewater surveillance dashboards.

To classify a system that is described to you, ask two questions in turn. First, who starts each report? If clinicians and laboratories report on their own initiative, usually because the law requires it, the system is passive; if public-health staff telephone, visit, or search records to find cases, it is active. Second, what is reported, and by whom? If a small, designated network of clinics or practitioners reports in a standard way, often with specimens for testing, the system is sentinel; if the data are pre-diagnostic signals such as chief complaints, school absences, or pharmacy sales, monitored in real time against a baseline, the system is syndromic. Once the type is named, its strengths and limitations follow from Table 2.1: a passive system, for example, is cheap, broad, legally mandated, and continuous, but it under-reports, and its timeliness and completeness depend on the reporting burden placed on clinicians. A legal requirement to report does not change who starts the report, so a laboratory that reports a notifiable result because the law requires it is still part of a passive system, not an active one. Many real products are hybrids, FluWatch among them (2.2.1), so classify the component that a question describes.

A fifth category, laboratory-based surveillance, is sometimes broken out separately because it sits beside (rather than under) the clinician's desk: provincial public-health labs aggregate isolates from clinical labs, perform serotyping or whole-genome sequencing, and feed the results into both passive and active systems. PulseNet Canada is the best-known example, the network that linked the cases of the 2008 Maple Leaf listeriosis outbreak across provinces by pulsed-field gel electrophoresis, while sequencing the genomes of two outbreak isolates during the outbreak characterised the strain (Gilmour et al., 2010).

Passive reporting of notifiable diseases is the backbone of Canadian surveillance, and its main limitation, under-reporting, follows directly from the chain of reports it depends on. Subsection 2.1 follows that chain.

2.1 The Reporting Flow and the Surveillance Pyramid

2.1.1 The Reporting Chain from Clinician to PHAC

For passive surveillance to function, every link in the chain has to do its part. The Canadian flow runs through four levels:

  1. A clinician or laboratory identifies a case of a notifiable disease (e.g., a positive shiga-toxin-producing E. coli culture or a clinical diagnosis of measles). Each province publishes its own list of conditions reportable under public-health legislation.
  2. The local Medical Officer of Health (MOH) or regional health authority receives the report (typically within 24 hours for urgent conditions, longer windows for routine ones). The MOH may immediately initiate case investigation, contact tracing, or public-health control measures.
  3. The provincial or territorial public-health authority (e.g., BCCDC, Public Health Ontario, the Quebec INSPQ) aggregates reports from the regional level, performs initial analysis, and shares anonymized aggregate data with the federal level.
  4. The Public Health Agency of Canada (PHAC) publishes nationally aggregated counts in CNDSS and feeds disease-specific products like FluWatch and the CCDSS. PHAC also reports onward under the WHO International Health Regulations (2005) for events of international concern; the post-SARS rationale for that regime is laid out by Heymann & Rodier (2004).

Two features of this flow deserve emphasis. First, public-health legislation is provincial in Canada, so the list of notifiable conditions and reporting timelines differ across provinces. A condition might be urgent-reportable in one province and routine in another. Second, the federal level receives aggregated, de-identified data only; PHAC cannot pull individual records. This federal-provincial division is one reason that the COVID-19 pandemic exposed gaps in real-time data sharing, because the legal architecture was not designed for the latency that a respiratory pandemic demands.

2.1.2 The Surveillance Pyramid: Why Passive Counts Are Low and Selected

The same chain explains why a passive system always counts fewer cases than actually occur. Each infection must pass a series of steps before it is counted, and some infections are lost at every step, a pattern often drawn as a surveillance pyramid with all infections at the base and reported cases at the tip:

  1. The person must become ill enough to notice. Many infections are mild or cause no symptoms at all.
  2. The person must seek care. Some people with symptoms stay at home, cannot reach a clinic, or do not think the illness is worth a visit.
  3. A specimen must be collected and tested. Clinicians do not order a stool culture or other test for every patient, and a test can miss the organism.
  4. The confirmed case must be reported to the medical health officer and then passed on through the province to PHAC. Some reports are late, incomplete, or never filed.

Because the chance of passing each step depends on the person, the losses are not random. Cases that present to the health system, get tested, and produce a positive laboratory result are systematically over-represented. Severe illness, young children whose parents take them to a doctor, and people who live near a clinic or laboratory are more likely to be tested and counted, and urban cases and cases in well-insured populations are more visible than rural or uninsured ones. Reported cases therefore over-represent severe, young, and well-served cases and under-represent mild, rural, and under-served ones. In the opening scenario, the four children whose paediatrician ordered stool cultures sit at the tip of the pyramid; classmates with milder illness who never saw a doctor would not appear in any report.

Why passive under-reporting is not random

A common student misconception is that under-reporting in passive surveillance just makes case counts smaller. It does, but it also biases who is counted. When you read a CNDSS rate, the denominator is the catchment population, but the numerator is selected. An answer about under-reporting should therefore make both points: name the levels of the pyramid where infections are lost, and explain why the cases that are counted are not a random subset of all infections.

The reports that survive this chain end up in national and provincial surveillance products, which subsection 2.2 describes.

2.2 Federal and Provincial Surveillance Products

Canada's surveillance products come in layers, and much of the skill in using them lies in matching a question to the product built to answer it. This subsection describes the federal products, the provincial layer in British Columbia, and the long-running data infrastructure that both read from.

2.2.1 Federal Products

PHAC operates several headline products. They are aggregated and curated, since the agency cannot dispense individual-level data, and each has a different cadence, scope, and intended audience.

Four products carry most of the national picture, and each answers a different question. The Canadian Notifiable Disease Surveillance System (CNDSS) gives long-run trends and comparisons between provinces for about 50 notifiable conditions, but it is too slow for real-time outbreak detection. FluWatch gives the weekly respiratory picture. The Canadian Chronic Disease Surveillance System (CCDSS) estimates the prevalence and incidence of chronic conditions from billing and hospital records. The Canadian Vital Statistics Death Database (CVSD) records every death in Canada with its cause. PHAC also runs a set of specialty systems for particular diseases and harms. Expand each panel below to see how each product is built, how often it reports, and what it is best used for.

CNDSS: Canadian Notifiable Disease Surveillance System ▼

The flagship passive system. Provinces and territories voluntarily submit annual data for ~50 nationally notifiable conditions (the list is harmonized but not identical to provincial lists). CNDSS feeds the agency's annual Notifiable Diseases Online reports and the underlying open-data tables on Open.canada.ca. Useful for long-run trends and inter-provincial comparisons; not useful for real-time outbreak detection, because the national data are compiled once a year and published with a lag of several years.

FluWatch: weekly respiratory virus surveillance ▼

A hybrid system, renamed FluWatch+ in 2024, when the Public Health Agency of Canada added national surveillance of RSV and SARS-CoV-2 to its influenza surveillance (Public Health Agency of Canada); this lesson uses the shorter name FluWatch. The Respiratory Virus Detection Surveillance System, a sentinel network of laboratories, reports tests and positive detections for up to 8 respiratory viruses, including influenza and SARS-CoV-2; FluWatchers volunteers report cough and fever each week online; sentinel hospital networks such as SPRINT-KIDS (15 paediatric hospitals in 8 provinces) follow severe cases; and provinces and territories report outbreaks in high-risk settings and hospitalizations. The weekly Canadian Respiratory Virus Surveillance Report is what most public-health communicators reach for during respiratory season.

CCDSS: Canadian Chronic Disease Surveillance System ▼

A federal-provincial collaboration that uses validated case definitions on top of health-administrative records (physician billing claims, hospital discharge abstracts) to estimate prevalence and incidence of chronic conditions like diabetes, hypertension, asthma, dementia, and ischaemic heart disease. CCDSS is one of the main sources of population-level chronic-disease data in Canada and supplies the chronic-disease tables of the agency's Health Infobase.

CVSD: Canadian Vital Statistics Death Database ▼

Not strictly a surveillance product but the spine of mortality-based surveillance. Statistics Canada compiles every death registered in Canada, with cause coded to ICD-10. CVSD enables life-expectancy estimates, cause-specific mortality trends, and excess-mortality analyses (most prominently used during COVID-19 to estimate pandemic burden).

Specialty surveillance systems ▼

PHAC also runs targeted systems for HIV (including the Canadian Perinatal Surveillance System branch), tuberculosis (CTBRS), antimicrobial resistance (CIPARS, CARSS), opioid-related harms, and others, alongside Canada's contribution to global digital surveillance, the Global Public Health Intelligence Network (GPHIN), described by Mykhalovskiy & Weir (2006) and complemented internationally by HealthMap (Brownstein, Freifeld, & Madoff, 2009). Each lives at canada.ca/en/public-health and is worth knowing exists.

FluWatch, which combines sentinel laboratories and hospitals, volunteer symptom reports and outbreak counts, is a good test of the classification in 1.2: because it is a hybrid, classify the component that a question describes. The national products give a country-level view built from aggregated data, and the provincial layer adds case-level detail.

2.2.2 The Provincial Layer in British Columbia

The federal products are valuable for the country-level view, but most case-level work happens provincially. In British Columbia, the BC Centre for Disease Control (BCCDC) is the analytic and operational arm of provincial public health, and most of its products are publicly available. As of October 2026, its data products worth bookmarking include the respiratory virus dashboards and reports (influenza, RSV, and SARS-CoV-2, updated weekly); the Communicable Disease Dashboard, which summarizes case data on reportable diseases in BC, among them enteric infections such as Salmonella, Campylobacter, STEC, and Listeria; the overdose response indicators, whose data on overdose deaths come from the BC Coroners Service; and the STI reports on gonorrhea, chlamydia, and syphilis. Cancer incidence is published separately by BC Cancer, which runs the BC Cancer Registry. Behind these products sits Panorama, BC's provincial public health information system, which is also used in several other provinces and territories, for communicable disease case and outbreak management and for immunization records.

2.2.3 The Long-Running Data Infrastructure

Most surveillance products do not generate their own data; they read from infrastructure that exists for other reasons. Vital statistics (births, deaths, marriages, divorces) are the oldest population health data in Canada, with continuous coverage since the late 19th century in some provinces; they are the denominator for life expectancy and the numerator for mortality surveillance. Cancer registries are among the few systems with active follow-up and complete case ascertainment: the Canadian Cancer Registry aggregates the provincial registries, and the BC Cancer Registry is the provincial source. Health-administrative data include the Discharge Abstract Database (DAD, hospital admissions), the National Ambulatory Care Reporting System (NACRS, ED visits), and provincial Medical Services Plan billing claims (MSP in BC); CCDSS and many academic studies are built on top of these.

Population health surveys add self-reported information: the Canadian Community Health Survey (CCHS) is the workhorse cross-sectional survey for self-reported health behaviours and outcomes, and the Canadian Health Measures Survey (CHMS) layers in physical measurement. Finally, wastewater-based surveillance has been a rapidly maturing infrastructure since 2020. PHAC and many provinces now sample wastewater for pathogens (SARS-CoV-2, influenza, mpox) as a non-clinical signal that is independent of who seeks testing, which makes it a partial answer to the selection described in 2.1.2.

Knowing these layers lets you match a need to a product and say what the product cannot tell you. This week's influenza activity points to FluWatch or the BCCDC respiratory dashboard; how common diagnosed diabetes is points to the CCDSS, which captures only people who have been diagnosed and billed; and excess deaths during a heat wave point to the CVSD. These sources differ in how quickly and how completely they capture events, and subsection 2.3 sets out the dimensions on which those differences can be judged.

2.3 Five Dimensions of Data Quality

Every surveillance product makes tradeoffs across five dimensions. When you read a public-health report, the report's author has implicitly resolved each one:

  1. Timeliness: how long from event to data product? Wastewater can be days. CNDSS can be months.
  2. Completeness: what fraction of true events are captured? STIs are notoriously incomplete; cancer registries are nearly complete.
  3. Representativeness: do the captured cases reflect the affected population? Sentinel networks usually do not.
  4. Sensitivity: will the system detect a true outbreak when it occurs? Syndromic surveillance is highly sensitive, passive surveillance often is not.
  5. Predictive value positive: when the system flags an event, is it real? It tends to trade off against sensitivity as the alert threshold moves, especially for syndromic systems, and it also depends on how often true events occur.

Sensitivity and predictive value positive are the two dimensions students most often mix up, because both describe how well a system's alerts match reality. They can both be written as proportions, but with different denominators. Sensitivity is the number of true outbreaks that the system flagged divided by all the true outbreaks that occurred. By this we mean that sensitivity starts from the real events and asks how many of them the system caught. Predictive value positive is the number of alerts that turned out to be real divided by all the alerts that the system raised. In this case the starting point is the alerts, and the question is how many of them deserved a response. For example, suppose a syndromic system raised 20 alerts in a year, and 5 of them turned out to be real outbreaks, while one further outbreak that year was never flagged. Its sensitivity would be 5 of 6 true outbreaks, which is high, but its predictive value positive would be only 5 of 20 alerts, because most of its alarms were false. The two numbers answer different practical questions: sensitivity tells you whether you can trust the system not to miss things, and predictive value positive tells you whether an alert on your desk is likely to be worth investigating.

The dimensions connect the earlier subsections. The four system types in 1.2 are, in effect, different settlements of these tradeoffs: a syndromic system buys timeliness and sensitivity at the cost of predictive value positive, which is why it raises many false alarms, while a sentinel system buys data quality at the cost of representativeness. The surveillance pyramid in 2.1.2 is a failure of completeness, and because the losses are not random it is also a failure of representativeness. To apply the dimensions to any source, ask each of the five questions in turn and look for the answer in the product's case definition and its published caveats. The exercise below does this with a real dashboard.

🔎 Hands-on: Explore a real dashboard

Pick one of the BCCDC dashboards listed in 2.2.2 (or a comparable PHAC product) and spend ten minutes with it. Locate (a) the most recent week's case count, (b) the underlying case definition, and (c) any explicit data-quality caveats the dashboard publishes. Note the date of the most recent update versus today's date: how big is the lag? Then judge the dashboard on the five dimensions above.

BCCDC data and reports · Health Infobase Canada

This section has defined surveillance and the loop it must close, described the four system types and the Canadian reporting flow, surveyed the federal and provincial products and the data infrastructure beneath them, and set out five dimensions of data quality. The key takeaways and knowledge check that follow review these points.

Key Takeaways

  • 1.1 Surveillance is more than collecting health data. Langmuir's working definition describes it as the ongoing, systematic collection, analysis, interpretation, and dissemination of health-related data for public-health action, and each part rules something out: ongoing separates it from a one-time study, systematic rules out anecdote, analysis and interpretation rule out a data warehouse that no one reads, and dissemination for action rules out findings that never reach the people who can use them. Surveillance “closes the loop” when the information comes back out as decisions, alerts, or programmes, such as case finding, control measures, or a public communication; data that are collected but never acted on are only bookkeeping. When you judge a described system, ask who looks at the data, how often, and what happens when something unusual appears. A common trap is to accept ongoing, systematic collection on its own, or the act of passing a file up the reporting chain, as surveillance; neither closes the loop until someone acts on the analysis.
  • 1.2 Surveillance systems come in four types, sorted mainly by who does the work of finding cases. In passive surveillance, clinicians and laboratories report notifiable diseases on their own initiative, usually because the law requires it (CNDSS is the Canadian example); in active surveillance, public-health staff telephone, visit, or search records to find cases (contact tracing during COVID-19); in sentinel surveillance, a small, designated network of clinics or practitioners reports in a standard way, often with specimens (FluWatch's sentinel laboratories and paediatric hospitals, and the Canadian Paediatric Surveillance Program); and in syndromic surveillance, pre-diagnostic signals such as chief complaints, school absences, or pharmacy sales are monitored in real time against a baseline (emergency-department chief-complaint monitoring, such as the system begun in Kingston, Ontario, in 2004). To classify a described system, ask two questions in turn: who starts each report, and what is reported and by whom. Its strength and limitation then follow from Table 2.1: passive is cheap, broad, legally mandated, and continuous but under-reports; active finds more cases with better data but is costly, narrow, and hard to sustain; sentinel gives richer, high-quality data at manageable cost but is not population-representative; and syndromic is fast but has low specificity, so it raises many false alarms. The most common error is to call a legally required laboratory report “active”, but a legal requirement does not change who starts the report; and because many real products, such as FluWatch, are hybrids, classify the component that the question describes.
  • 2.1 Passive surveillance relies on a chain of reports: a clinician or laboratory identifies a notifiable disease and reports it to the medical health officer, the provincial public-health authority (in BC, the BCCDC) aggregates the reports, and PHAC publishes national counts in CNDSS. Each province sets its own list of notifiable conditions and reporting timelines, and PHAC receives only aggregated, de-identified data. Because infections drop out at every level of the surveillance pyramid (the illness is mild or causes no symptoms, the person does not seek care, no specimen is tested or the test misses the organism, or the confirmed case is reported late or not at all), the reported count is always smaller than the true number of infections. When a question asks you to explain the undercount “systematically”, walk up the pyramid and name at least three distinct places where infections are lost. Then make the second point, which is easy to forget: the losses are not random, because severe cases, young children whose parents take them to a doctor, and people who live near a clinic or laboratory are more likely to be tested and counted, so reported cases over-represent severe, young, and well-served cases and under-represent mild, rural, and under-served ones.
  • 2.2 Canada's surveillance products come in layers, and much of the skill lies in matching a question to the product built to answer it. At the federal level, PHAC publishes aggregated products: CNDSS for long-run trends and comparisons between provinces in about 50 notifiable conditions (too slow for real-time outbreak detection), FluWatch for the weekly respiratory picture from sentinel laboratories and hospitals, volunteer symptom reports, and outbreak counts, the CCDSS for the prevalence and incidence of chronic conditions such as diabetes and dementia, estimated by applying case definitions to billing and hospital records, and the CVSD, Statistics Canada's death database, for cause-specific mortality, life expectancy, and excess deaths. The BCCDC dashboards and reports add provincial detail on, for example, respiratory viruses, reportable communicable diseases, overdose deaths, and sexually transmitted infections, and BC Cancer reports cancer incidence. Most of these products do not create their own data but read from long-running infrastructure: vital statistics, cancer registries, health-administrative data (DAD, NACRS, and MSP billing), national surveys such as the CCHS, and wastewater sampling. When a question describes a need, such as this week's influenza activity, how common diagnosed diabetes is, or excess deaths during a heat wave, name the matching product and say what it cannot tell you.
  • 2.3 Every data source trades some qualities against others, and five questions let you judge any of them. Timeliness asks how long it takes from the event to the data (days for wastewater, weeks to months for CNDSS); completeness asks what fraction of true events are captured (nearly all for cancer registries, far fewer for sexually transmitted infections); representativeness asks whether the captured cases reflect the affected population (sentinel networks usually do not); sensitivity asks whether the system will detect a true outbreak when it occurs; and predictive value positive asks whether a flagged event is real. Sensitivity and predictive value positive use different denominators and are easy to confuse: sensitivity divides the outbreaks the system flagged by all the outbreaks that occurred, while predictive value positive divides the real alerts by all the alerts it raised, so a syndromic system can be highly sensitive and still raise many false alarms. In an answer, identify the dimension that limits the question being asked and explain why for that source, rather than simply listing all five.
Knowledge Check: this section

Question 1: A health unit asks every school in its region to email a weekly count of absent students. The counts are saved in a shared spreadsheet, but no one is assigned to review them. After a gastroenteritis outbreak affects three schools, staff notice that the spreadsheet showed a sharp rise in absences two weeks before the outbreak was reported. A manager says, "At least we already have a surveillance system for school absences." Using Langmuir's working definition, which response is best?

Ongoing, systematic collection is only the first part of Langmuir's definition. Without analysis, interpretation and dissemination to people who can act, collection is bookkeeping, not surveillance.
Passing data up the reporting chain is not public-health action. The loop closes only when analysed information returns as decisions, alerts or programmes, which never happened here.
Langmuir's definition requires collection, analysis, interpretation and dissemination for action; the counts were stored but never reviewed, so the loop was not closed and the rise went unused.
Surveillance is not limited to notifiable diseases. School absences are a recognised syndromic signal; the failure here is that nobody analysed or acted on the counts.
Correct answer: the manager is wrong, because the loop is not closed. The information needed is what happens to the counts after they are collected: they are stored, but nobody analyses them or acts on them. Langmuir's working definition has four parts (the ongoing, systematic collection, the analysis, the interpretation, and the dissemination of health data for public-health action), and surveillance closes the loop only when the information comes back out as decisions, alerts, or programmes. The first option is tempting because ongoing, systematic collection is the first part of the definition, but collection alone is bookkeeping; sending the file up the reporting chain is not action either, and school absences are a recognised syndromic signal. (Section 1, 1.1 Defining Surveillance and Closing the Loop; Objective 1.1)

Question 2: During a summer of enteric illness, a health authority can draw on four sources. (1) Clinicians and laboratories report every confirmed case of salmonellosis to the medical health officer, as the provincial public-health act requires. (2) Daily sales of anti-diarrhoeal medicines at community pharmacies are monitored, and an automatic alert is raised when sales rise above the usual level. (3) A designated group of 25 paediatricians, chosen to cover the province, reports every month each child they have seen with haemolytic uraemic syndrome, using a standard form. (4) Public-health staff review the admission records of every hospital in the region each week to find patients with salmonellosis. Which classification is correct?

Ask who starts each report and what is reported: mandated clinician reports are passive, pharmacy sales against a baseline are syndromic, the designated paediatrician network is sentinel, and staff searching records is active.
This swaps sentinel and syndromic. Monitoring medicine sales against a baseline is a pre-diagnostic signal (syndromic); a small designated network reporting on a standard form is sentinel.
A legal duty to report does not make (1) active, because clinicians still start the report. In (4) public-health staff go looking for cases, which is what makes a system active.
The paediatricians are a small, designated network reporting in a standard way, which is sentinel. Public-health staff reviewing hospital records to find cases is active surveillance.
Correct answer: (1) passive, (2) syndromic, (3) sentinel, (4) active. Ask two questions of each source: who starts each report, and what is reported and by whom. In (1), clinicians and laboratories report on their own initiative because the law requires it, so the system is passive; in (4), public-health staff go looking for cases in hospital records, so it is active; (3) is a small, designated network that reports in a standard way, so it is sentinel; and (2) monitors a pre-diagnostic signal against a baseline, so it is syndromic. The most common error is to call (1) active because reporting is mandatory, but a legal requirement does not change who starts the report. Once the type is named, its strength and limitation follow from Table 2.1: passive surveillance, for example, is cheap, broad, and continuous, but it under-reports. (Section 1, 1.2 The Four Types of Surveillance System, Table 2.1 and the paragraph after it; Objective 1.2)

Question 3: Last year, 120 cases of campylobacteriosis were reported through the notifiable-disease system in a largely rural health region. Two-thirds of the reported cases were young children or people admitted to hospital. A colleague writes: "About 120 people in the region were infected last year, and young children and hospitalised patients are the groups most affected." Which critique of this statement is correct?

A legal duty applies only to cases that are diagnosed and confirmed. Infections lost before that point (no symptoms, no visit, no test) never reach the count, and the losses distort the case mix.
The surveillance pyramid points the other way: infections are lost at each step before a report, so the count is lower than the number of infections, not higher.
The undercount is right, but the missed infections are not a random subset. Severity, age and access to care affect who is tested and reported, so the case mix is distorted.
Infections are lost at each step of the pyramid, so the count is too low; because severe illness and young children are more likely to be tested and reported, the losses are not random and the case mix is distorted.
Correct answer: the count is too low and the case mix is distorted. The information needed is how a case reaches the count: the infected person must notice the illness, seek care, and have a specimen tested and found positive, and the clinician or laboratory must then report to the medical health officer, who passes the case to the province and on to PHAC. Infections are lost at every one of these steps, so the reported count is smaller than the number of infections. Because the chance of passing each step depends on severity, age, and access to care, the losses are not random: severe cases and young children who are taken to a doctor are over-represented, and mild, rural, and under-served cases are missed, so the large share of children and hospitalised patients partly reflects who gets tested. The third option is tempting because it gets the undercount right, but it assumes the missed infections are a random subset, which is the misconception the callout in 2.1.2 warns against. (Section 1, 2.1 The Reporting Flow and the Surveillance Pyramid, including the callout on why passive under-reporting is not random; Objective 2.1)

Question 4: An analyst is assembling data for four briefing questions. Which pairing of question and Canadian surveillance product is correct?

The CCDSS estimates chronic conditions from health-administrative data. Weekly influenza activity across Canada is the job of FluWatch.
The CCDSS applies validated case definitions to physician billing and hospital discharge data to estimate prevalence and incidence of chronic conditions such as diabetes, by province over time.
FluWatch tracks weekly influenza activity. Deaths and excess mortality come from the Canadian Vital Statistics Death Database compiled by Statistics Canada.
The CVSD records deaths, not reported infections. Long-run trends in notifiable infections such as gonorrhoea come from the CNDSS.
Correct answer: diagnosed diabetes by province, from the CCDSS. Match each question to what each product measures. The CCDSS applies validated case definitions to health-administrative data (physician billing claims and hospital discharge abstracts) to estimate the prevalence and incidence of chronic conditions such as diabetes, so it fits the diabetes question. Weekly influenza activity comes from FluWatch, deaths and excess mortality from the CVSD compiled by Statistics Canada, and long-run trends in notifiable infections such as gonorrhoea from CNDSS, so each of the other pairings uses a product built for a different question. (Section 1, 2.2.1 Federal Products and 2.2.3 The Long-Running Data Infrastructure; Objective 2.2)

Question 5: A syndromic system that monitors emergency-department chief complaints raised 20 alerts for gastrointestinal illness last year, and investigation found a real outbreak behind 4 of them. Over the same year, 5 gastrointestinal outbreaks occurred in the region, and the system flagged 4 of them. A colleague says, "The main weakness of this system is its low sensitivity." Which response is correct?

4 of 20 alerts is the predictive value positive, the share of alerts that were real. Sensitivity divides outbreaks flagged by all outbreaks that occurred: 4 ÷ 5 = 80%.
The system flagged 4 of the 5 outbreaks that occurred, so it missed only one. Its weakness is false alarms, not missed outbreaks.
Sensitivity uses all true outbreaks as the denominator (4 ÷ 5 = 80%), while predictive value positive uses all alerts (4 ÷ 20 = 20%); the system catches outbreaks but raises many false alarms.
Arriving before laboratory confirmation is the main strength of syndromic surveillance, not a weakness. The low share of real alerts is the problem here.
Correct answer: sensitivity is high (80%) and predictive value positive is low (20%). Two different denominators are needed. Sensitivity asks whether the system catches true outbreaks when they occur, so it divides the outbreaks flagged by all outbreaks that occurred: 4 ÷ 5 = 80%. Predictive value positive asks whether an alert is real, so it divides the real alerts by all alerts: 4 ÷ 20 = 20%, which means four of every five alerts were false alarms. The first option is the most tempting error, because it computes predictive value positive (4 of 20 alerts) and calls it sensitivity; early arrival of the signal is a strength of syndromic surveillance, not a weakness. (Section 1, 2.3 Five Dimensions of Data Quality, the list of five dimensions and the paragraph after it; Objective 2.3)

Question 6: Pertussis (whooping cough) is a notifiable disease in British Columbia. In one health region, clinicians and laboratories report every laboratory-confirmed case to the medical health officer, as the provincial public-health act requires, and the reports are passed on to the province and to PHAC. How should this system be classified, and what are one strength and one limitation of that type?

A legal requirement does not change who starts the report. Clinicians and laboratories report on their own initiative and no staff go looking for cases, so the system is passive.
Clinicians and laboratories start each report, which makes the system passive; Table 2.1 lists its strengths as cheap, broad, mandated and continuous, and its main limitation as under-reporting.
Sentinel systems use a small designated network. Here every clinician and laboratory in the region reports, so restricting to laboratory-confirmed cases does not make it sentinel.
Syndromic surveillance monitors pre-diagnostic signals against a baseline. These are laboratory-confirmed diagnoses reported by clinicians and laboratories, which is passive surveillance.
Correct answer: passive surveillance, which is cheap and broad but under-reports. The information needed is who starts each report: here clinicians and laboratories report on their own initiative, and no public-health staff go looking for cases, so the system is passive. Its strengths are low cost, broad coverage, a legal mandate, and continuous operation; its main limitation is under-reporting, with timeliness and completeness that depend on the reporting burden placed on clinicians. The first option is tempting because the law requires the reports, but a legal duty to report does not change who starts the report, so it does not make a system active. (Section 1, 1.2 The Four Types of Surveillance System, Table 2.1 and the paragraph after it; Objective 1.2)

Question 7: In the same region, 14 pertussis cases were reported in March, against a usual 3 a month, and on average each report reached the medical health officer 12 days after the person's cough began. Which data-quality dimension does the 12-day delay describe, and which response would close the loop on the March reports?

The delay from cough onset to report is timeliness, not completeness. Filing reports with the province completes the chain but is bookkeeping, not action that closes the loop.
The delay concerns timeliness, not representativeness, and waiting for an annual report would make the response even later, when contacts may already be ill.
Predictive value positive concerns whether a signal is real. The 12-day delay measures timeliness, and re-confirming confirmed cases would delay action further.
Timeliness is the time from the event to data reaching public health; closing the loop means acting on the rise, for example by alerting clinicians and contacting the close contacts of each case.
Correct answer: timeliness, with a response that returns the information to people who can act on it. The information needed is what the delay measures: the time from the event (the start of the cough) to the data reaching public health, which is the definition of timeliness. With a 12-day delay, close contacts may already be ill before the reports arrive, which makes prompt action more important. Closing the loop means that the analysed reports come back out as decisions, alerts, or programmes: recognising that 14 cases is well above the usual 3, then alerting clinicians and reaching the contacts of each case. The first option is tempting because it completes the reporting chain, but passing a file up the chain is bookkeeping, not action, so it does not close the loop. (Section 1, 1.1 Defining Surveillance and Closing the Loop and 2.3 Five Dimensions of Data Quality; Objectives 1.1 and 2.3)
Section 2 of 5

Detecting and Investigating Outbreaks

⏱ Estimated reading time: 45 minutes
Section 2 of 5

Detecting and Investigating Outbreaks

Definitions, the CDC 10-step framework, the Canadian protocol, and one Canadian outbreak followed step by step.

Vocabulary

Cluster, outbreak, epidemic, pandemic

Cluster

Aggregation in time or space. May or may not exceed the expected baseline.

Outbreak

Cluster judged to exceed the baseline by a public-health authority. Partly statistical, partly operational.

Epidemic

Large-scale outbreak, often multi-jurisdictional. Often used interchangeably with outbreak.

Pandemic

Worldwide geographic spread. A geographic claim, not a severity claim.

CDC 10-step framework, part 1

Steps 1 through 6

  • Step 1: Prepare for fieldwork (authority, supplies, liaisons)
  • Step 2: Establish the outbreak exists vs. baseline
  • Step 3: Verify the diagnosis (review charts, talk to clinicians)
  • Step 4: Construct a working case definition
  • Step 5: Find cases; build the line list
  • Step 6: Descriptive epi covering person, place, time and the epi curve

The epidemic curve shape (point source, continuous, or propagated) constrains your hypotheses about the exposure window.

The epi curve and steps 7–10

Reading the curve, then testing hypotheses

Point source Continuous source Propagated Time (days from exposure)
Canadian FIORP

Three federal partners, one protocol

PHAC

Epidemiology and surveillance lead. Operates PulseNet Canada for WGS-based cluster detection.

CFIA

Food-safety investigations, traceback from grocery purchases, and product recalls.

Health Canada

Risk assessment of contaminated products; health-impact guidance.

FIORP defines escalation triggers and the Outbreak Investigation Coordinating Committee for multi-jurisdictional events.

The outbreak

2017–18 E. coli O157:H7, romaine lettuce

Scale

42 confirmed cases · 5 eastern provinces.
25 US cases · 15 states.
CFR: about 2% (1 death among 42 cases).

Three teaching points

1. Genetic fingerprinting linked scattered cases into one cluster.
2. Exposure among cases needed a comparison group.
3. Precautionary advice came before the contamination was traced.

Steps 2–3: The genomic signal

How PulseNet Canada made the cluster visible

ON QC NB NS NL Before subtyping: scattered background noise

Matching PFGE patterns, confirmed by whole-genome sequencing (WGS), revealed a shared phylogenetic neighbourhood. Same cases, newly visible as a cluster.

Steps 6–10

Descriptive epi, comparison groups, precautionary advice

Epi curve shape: onsets spread over several weeks, consistent with a continuous common-source outbreak from an ongoing supply-chain contamination.

Exposures: most Canadian cases ate romaine; in the US, 55% of cases against 46% of healthy people in a survey (not significant), so only leafy greens were implicated.

Advice: on 21 Dec 2017, consider other lettuce; withdrawn on 10 Jan 2018, when the outbreak appeared to be over.

Outcome

All food samples tested negative, and the cause of contamination was never identified. Precautionary action was still the right call.

Line-list analysis

What a line-list analysis involves

Attack Rate (closed cohort)
\[ \color{#0B7B6B}{\text{AR}} = \frac{\color{#C2410C}{\text{cases among exposed}}}{\color{#6D28D9}{\text{total exposed}}} \]
AR attack ratecases among exposed ill people who ate the foodtotal exposed everyone who ate the food
Risk Ratio
\[ \color{#0B7B6B}{RR} = \frac{\color{#C2410C}{AR_{\text{exposed}}}}{\color{#6D28D9}{AR_{\text{unexposed}}}} \]
RR risk ratioARexposed attack rate among those who ate the foodARunexposed attack rate among those who did not

If two foods both show high RR and many guests ate both, stratify: compute the RR for Food A within strata of Food B. The food whose RR holds across both strata is the source.

Carry forward

What to take into the next section

  • An outbreak is a cluster judged to exceed the expected baseline, partly by statistics and partly by judgement.
  • The ten steps overlap in practice, and control measures often begin before the analytic study ends.
  • Speed and accuracy pull in opposite directions, and the costs of error rarely fall on the same people.
  • When the population at risk cannot be listed in full, the investigation has to sample.

Overview

Surveillance is meant to detect signals, and outbreak investigation is the structured response when a signal turns into a problem. This section sets out what counts as an outbreak, describes the CDC 10-step framework that organizes an investigation, and shows how to read the epidemic curve that the framework produces. It then introduces the Canadian protocol for foodborne outbreaks and follows the 2017–18 romaine lettuce E. coli O157:H7 outbreak through the same ten steps. It closes with the line-list analysis, from attack rates to stratified risk ratios, that sits at the centre of most investigations.

Learning Objectives

  • 3.1 Distinguish cluster, outbreak, epidemic, and pandemic, and explain why declaring an outbreak is partly a statistical and partly an operational judgement.
  • 3.2 Describe the CDC 10-step outbreak investigation framework, including the three parts of a working case definition, and identify which steps typically run in parallel.
  • 3.3 Read an epidemic curve, name its shape (point source, continuous common source, or propagated), and use the incubation period to estimate the likely exposure window.
  • 3.4 Describe the Canadian Foodborne Illness Outbreak Response Protocol (FIORP) and the roles of PHAC, CFIA, and Health Canada.
  • 3.5 Follow the 2017–18 romaine lettuce outbreak through the ten steps and explain the roles of whole-genome sequencing and of the comparison group.
  • 4.1 Calculate attack rates and risk ratios for several foods from a line list, identify the likely vehicle, and explain how stratification separates two suspect foods.

3.1 Cluster, Outbreak, Epidemic and Pandemic

The textbook definition of an outbreak is the occurrence of more cases of a disease than expected in a given population, place, or time. Each phrase is doing work. “More than expected” presupposes a baseline: the local 5-year average for influenza in the same week, or the seasonal threshold from a regression model. “Population, place, time” says outbreaks are local: 30 cases of Campylobacter across Canada in a week is not unusual; 30 cases at one wedding is. Four related terms describe an excess of cases at different scales, and they need to be used precisely.

Four working terms you need to use precisely

  • Cluster: an aggregation of cases in time and/or space that may or may not be statistically unusual. Clusters are flagged by surveillance and triaged for further investigation.
  • Outbreak: a cluster judged to exceed the expected baseline. The threshold is decided by the public-health authority, not by the data alone.
  • Epidemic: a large-scale outbreak, often across multiple jurisdictions; in international usage often synonymous with outbreak.
  • Pandemic: an epidemic with worldwide geographic spread. The WHO declares pandemics; severity is a separate dimension and is not part of the definition.

A common student error is to treat “pandemic” as “severe outbreak”: it is a geographic claim, not a severity claim.

The step from cluster to outbreak shows why the decision to call something an outbreak is partly statistical and partly operational. Statistically, you can compare current counts to a baseline distribution and apply a threshold (e.g., the upper 95% confidence limit of the 5-year mean). Operationally, an outbreak declaration mobilizes resources, triggers a coordinated response, and may require public communication, and authorities are reasonably cautious about both over- and under-calling. In a scenario, look first for the baseline and then for who has made the judgement: the four children with bloody diarrhoea in Section 1 form a cluster, and they become an outbreak only when the health authority judges that they exceed what is expected for that organism in that catchment.

Once an outbreak has been declared, the investigation follows a standard sequence of steps, which subsection 3.2 sets out.

3.2 The CDC 10-Step Investigation Framework

The reference framework most North American epidemiologists learn is the CDC's 10-step process, articulated in the modern era by Reingold (1998) and elaborated in the CDC Field Epidemiology Manual. The ten steps move from establishing that there is something to investigate (Step 1, prepare for fieldwork; Step 2, establish the existence of an outbreak; Step 3, verify the diagnosis), through assembling and describing the cases (Step 4, construct a working case definition; Step 5, find cases systematically and build a line list; Step 6, perform descriptive epidemiology), to explaining and acting (Step 7, develop hypotheses; Step 8, evaluate them with an analytic study; Step 9, refine them with additional studies; Step 10, implement control and prevention and communicate findings).

Two points about the framework matter most. First, a working case definition always has three parts: clinical criteria; restrictions by person, place, and time; and a level of certainty (suspect, probable, or confirmed). When you check a draft definition, look for all three parts, and keep the suspected exposure out of it, because the analytic study in Step 8 must compare people who did and did not have that exposure. Second, the steps look orderly on paper but in practice you often work several at once and revisit earlier ones as new information arrives: communication runs throughout, the case definition is revised as the investigation evolves, and control measures often begin before the analytic study is finished.

Interactive 2.1 retells the 1976 outbreak at the American Legion convention in Philadelphia in eight scenes, each illustrating a stage of the framework. Step through it with Next to see the steps in sequence in a real investigation.

▸ Interactive 2.1: INTERACTIVE STORY: LEGIONNAIRES' 1976 Open full screen ↗

Walk through the most famous post-war outbreak investigation, scene by scene. Next ▶ advances.

An 8-scene retelling of the 1976 American Legion convention outbreak in Philadelphia, illustrating the CDC's 10-step framework: detection, case definition, descriptive epi, hypothesis generation, analytic study, environmental investigation, agent identification (a brand-new bacterium), and control measures.

Expand each of the ten panels below for what the step involves in practice, including the pitfalls new investigators meet, the two analytic designs used in Step 8, and the kinds of control measure used in Step 10.

Step 1: Prepare for fieldwork ▼

Before you arrive, you confirm authority and roles, brief on the suspected etiology, gather supplies (case-report forms, lab kits, PPE), and identify local liaisons (MOH, environmental health, lab). Preparation is the step new investigators most often skip and most often regret.

Step 2: Establish the existence of an outbreak ▼

Compare current counts to a baseline. If the baseline is unstable (small denominators, seasonal variation), state how you constructed it. Ruling out artefact, whether a new lab test, a reporting policy change, or a clinician on a reporting kick, is part of this step.

Step 3: Verify the diagnosis ▼

Talk to clinicians, review charts, check that lab results are correctly attributed. A pseudo-outbreak driven by a contaminated lab reagent or a misclassification is not unheard of.

Step 4: Construct a working case definition ▼

A case definition has three parts: clinical criteria (symptoms, signs, lab tests), person/place/time restrictions (e.g., attendees of the August 12 potluck), and a level of certainty (suspect, probable, confirmed). You will revise it as the investigation evolves, which is normal.

Step 5: Find cases systematically and record information ▼

Active case finding (chart review, asking clinicians, contacting attendees of the implicated event) yields a line list, with one row per case carrying demographic, clinical, exposure, and outcome variables. The line list is the working dataset for everything that follows.

Step 6: Perform descriptive epidemiology ▼

The classic person, place, time triad. The flagship visualization is the epidemic curve (epi curve), a histogram of case counts by date of symptom onset, and its shape (point-source vs propagated) constrains your hypotheses about the exposure window. Subsection 3.3 explains how to read the curve and how to work back from it to the likely time of exposure.

Step 7: Develop hypotheses ▼

From the descriptive epi (and from open-ended interviews of cases) you generate plausible exposures: a specific food, a specific water source, a specific event, a specific procedure. Good hypotheses are testable with the data you have or can collect.

Step 8: Evaluate hypotheses with an analytic study ▼

The two workhorse designs in outbreak settings are the retrospective cohort (when you can enumerate everyone who attended an event, e.g., the wedding-guest list) and the case-control study (when you cannot). The analysis of a cohort uses attack rates (the proportion of people in a group who became ill), 2×2 tables of exposure by illness, and risk ratios (the attack rate in the exposed divided by the attack rate in the unexposed), and the line-list analysis later in this section shows how they are used in an investigation.

Step 9: Refine hypotheses and carry out additional studies ▼

The first analytic pass often points to several plausible exposures. Environmental sampling, traceback investigations (where did the implicated food come from?), and lab characterization (genome-typing of isolates) sharpen the inference.

Step 10: Implement control and prevention; communicate findings ▼

Control measures (recalling a product, closing a venue, prophylaxis, isolation) often happen before Step 8; the precautionary principle does not require you to wait for a p-value. Communication runs throughout: with the public, with affected communities, with policymakers, and through the final outbreak report.

Acting before the analysis is complete, as Step 10 allows, reflects a standing tension between two pressures. Speed, since every day of delay can mean more illness, pushes you toward early hypotheses and precautionary control measures. Accuracy, since falsely accusing a food product or a venue has real costs, pulls in the other direction. Experienced investigators learn to act on confident-enough evidence, communicate uncertainty honestly, and revise control measures as data evolve. The skill is partly statistical and partly ethical: who bears the cost of being wrong in either direction is rarely symmetric.

Of the ten steps, Step 6 deserves separate treatment, because its main product, the epidemic curve, constrains every hypothesis that follows. Subsection 3.3 shows how to read it.

3.3 Reading an Epidemic Curve

Step 6, perform descriptive epidemiology, is the classic person, place, and time triad. Its flagship visualization is the epidemic curve (epi curve), a histogram of case counts by date of symptom onset. The shape of the curve points to how people were exposed, and Figure 2.1 shows the three idealized shapes. A sharp rise to a single peak that falls away within about one incubation period suggests a point source, one brief shared exposure such as a single meal. A broad plateau suggests a continuous common source that is still active. Successive peaks spaced about one incubation period apart suggest propagated, person-to-person spread.

Three epidemic-curve shapes: point source, continuous common source, and propagatedPoint sourcesingle sharp peakone brief exposureContinuous sourcebroad plateauongoing exposurePropagatedsuccessive wavesperson-to-person spread
Figure 2.1. Idealized epidemic-curve shapes. A point source produces a single sharp peak about one incubation period wide; a continuous common source produces a broad plateau; a propagated outbreak produces successive waves spaced roughly one incubation period apart as cases infect others.

When the curve has the single sharp peak of a point source, the incubation period of the suspected organism can be used to work backward from the onsets to the time of exposure. First, choose a time unit shorter than one incubation period (hours for an agent with an incubation of 6 to 12 hours, days for one with an incubation of several days), so that the shape is not smoothed away. Second, find the peak and count back by the average incubation period: the exposure should lie about one average incubation period before the peak. Third, check the edges: the first case should fall no earlier than the minimum incubation period after the suspected exposure, and the last case of the first wave no later than the maximum incubation period after it. For example, if an organism has an average incubation of 2 days and a range of 1 to 5 days, and a church supper was held on a Sunday, a curve whose first case begins on Monday, which peaks on Tuesday, and whose last case begins on Thursday fits a single exposure at the supper. The onsets are spread over several days because incubation periods vary, not because people infected one another. A curve with a second peak about one incubation period after the first suggests that the first cases then infected others (propagated spread), which does not by itself rule out a point source for the first wave.

The framework and the epidemic curve describe an investigation in general terms. For foodborne outbreaks that cross jurisdictions, Canada also has a formal protocol that assigns each task to a partner, and subsection 3.4 describes it.

3.4 The Canadian Foodborne Illness Outbreak Response Protocol

For foodborne outbreaks specifically, Canada operates under the Foodborne Illness Outbreak Response Protocol (FIORP), summarised by Vik & Hexemer (2014), which formalizes the roles of the federal partners and provincial/territorial public-health authorities. Three federal partners share the load. PHAC is the epidemiology and surveillance lead, including PulseNet Canada, the lab network that does whole-genome sequencing of bacterial isolates. The Canadian Food Inspection Agency (CFIA) leads food-safety investigations, traceback, and product recalls. Health Canada leads the risk assessment of contaminated products and provides health-impact guidance. In a scenario, match each task to its partner: sequencing isolates and linking cases is PHAC, tracing a product back and recalling it is CFIA, and judging the health risk of a product is Health Canada.

FIORP defines escalation triggers (when a multi-jurisdictional outbreak exists), establishes an Outbreak Investigation Coordinating Committee (OICC) for inter-provincial events, and lays out communication protocols. The 2008 Maple Leaf listeriosis outbreak (the deli-meat-associated Listeria monocytogenes outbreak that killed 22 Canadians) is the case that drove the modern revisions to FIORP and to the supporting infrastructure of PulseNet. Subsection 3.5 follows a more recent multi-jurisdictional outbreak through the ten steps and shows the FIORP partners at work.

3.5 Case Study: The 2017–18 Romaine Lettuce E. coli O157:H7 Outbreak

Between November 2017 and February 2018, PHAC and US CDC investigated a multi-jurisdictional outbreak of E. coli O157:H7 infections that ultimately involved 42 confirmed cases across 5 Canadian provinces (and 25 cases across 15 US states). All five affected Canadian provinces were in eastern Canada (Ontario, Quebec, New Brunswick, Nova Scotia, and Newfoundland and Labrador); western provinces were not involved. One death was reported among the 42 Canadian cases, a case fatality rate of about 2% (roughly 2 in every 100 diagnosed cases died). The outbreak is a useful teaching case because it shows the full FIORP machinery in motion and makes the role of whole-genome sequencing visible. The paragraphs below follow it through the steps of 3.2.

Steps 2 and 3, establishing the outbreak and verifying the diagnosis. PulseNet Canada flagged a cluster of E. coli O157:H7 isolates with matching pulsed-field gel electrophoresis (PFGE) patterns, later confirmed by whole-genome sequencing (WGS) to share a tight phylogenetic neighbourhood. PFGE compares the pattern of fragments produced when the bacterial DNA is cut up, while WGS reads the organism's genetic code in far greater detail. By a tight phylogenetic neighbourhood we mean that the isolates differed from one another by so few genetic changes that they very likely descended from a single recent source, much as a group of close cousins can be traced to the same grandparents. The genetic match is what linked cases in different provinces, and in the United States, into one investigable outbreak rather than a scattered set of unrelated cases. Because E. coli O157:H7 infections occur every week across Canada, a few extra cases in one province need not stand out on their own; laboratory subtyping shows which of them belong together. This is the laboratory-based surveillance of Section 1 doing the work of Step 2.

Steps 4 to 6, case definition and descriptive epidemiology. A case definition for an outbreak like this one combines a laboratory criterion, such as an isolate that matches the outbreak strain, with restrictions by person, place, and time, and it can separate confirmed from probable cases. Here the cases lived in five eastern provinces and became ill in November and early December 2017, with no onsets after 12 December (Public Health Agency of Canada, 2018). The definition therefore has the three parts set out in 3.2: clinical and laboratory criteria, restrictions by person, place, and time, and levels of certainty. The descriptive analysis (epi curve, geographic distribution, age and sex distribution) showed that onsets did not cluster around a single day, as they would after one shared meal, but continued over several weeks as contaminated lettuce kept reaching stores and being eaten. That pattern is typical of a continuous common-source outbreak driven by an ongoing contaminated supply chain, not a single point exposure (3.3).

Steps 7 and 8, developing and testing hypotheses. Interviews with the cases found that most of them had eaten romaine lettuce before they became ill, which pointed the Canadian investigation to romaine. A share of cases who ate a food means little on its own, because many healthy people eat it too, so it has to be set beside a comparison group. In the related US outbreak, 55% of the ill people interviewed had eaten romaine, but 46% of healthy people in a food survey reported eating it, a difference that was not statistically significant, so US investigators could implicate only leafy greens in general (Centers for Disease Control and Prevention, 2018). When healthy controls cannot be listed and interviewed quickly, investigators have faster options: comparing the cases with a background survey of what healthy people eat, as in the US investigation, or a case-case design, in which the comparison group is made up of people who had the same kind of illness but were not part of the outbreak, such as people with sporadic STEC infections reported earlier, whose questionnaires are already on file. Either way, the investigators ask whether the outbreak cases ate a food more often than the comparison group did. These designs are pragmatic alternatives to a traditional case-control study when controls are hard to recruit during a fast-moving foodborne investigation, because finding, contacting, and interviewing healthy people takes weeks that an ongoing outbreak does not allow. The cost is some validity. Comparison cases in a case-case design are themselves ill, so they may share some exposures with the outbreak cases, and survey respondents may differ from the cases in when and where they were asked, so each comparison is less clean than one with healthy controls recruited at the same time. You therefore trade some validity for considerable speed. CFIA led the food-safety investigation in parallel, and all the food samples tested were negative.

Steps 9 and 10, refinement, control, and communication. On 21 December 2017, PHAC advised people in the affected provinces to consider eating other types of lettuce instead of romaine. On 10 January 2018 it reported that the outbreak appeared to be over and withdrew that advice, and it closed the investigation on 9 February 2018; CDC gave no advice to avoid a particular food in the United States, because it had not identified one. Romaine lettuce was identified as the source of the Canadian outbreak, but the cause of its contamination was never found, a sobering reminder that even good investigations sometimes end without the closure of a single confirmed source. The advice was issued before the contamination was traced, an example of the precaution described in 3.2: precaution is a defensible public-health stance when downside asymmetry favours acting early.

Three teaching points come out of this case, each tied to an earlier subsection. First, laboratory subtyping linked cases in different provinces, and in two countries, into one cluster. Second, the share of cases who ate a food has to be compared with a comparison group, and when healthy people cannot be listed quickly, faster comparisons trade some validity for speed. Third, control measures came before the cause of contamination was found. The romaine analysis rested on comparing groups; subsection 4.1 shows how the same kind of comparison is made from a line list.

4.1 From Line List to Risk Ratio

Most investigations rest on a line list, which holds one row per person, with columns for illness, date of onset, and exposures. Three quantities come directly from it. The attack rate is the proportion of an exposed group who became ill. For each suspect food, a 2×2 table compares the attack rate among people who ate the food with the attack rate among people who did not, and the ratio of the two attack rates is the risk ratio. A risk ratio near 1 suggests that the food is unrelated to illness, and a large risk ratio marks the food as a suspect. Always divide the number ill by the size of each group before comparing, and give two reasons when you name the likely vehicle: it has the largest risk ratio, and most of the cases ate it.

When many people ate two foods together, both foods can show a high crude risk ratio, because each food carries part of the other's association. A crude risk ratio is one calculated from all the guests at once, without taking any other food into account. If most of the people who ate the potato salad also ate a contaminated ham, the potato salad eaters will have a high attack rate even though the salad itself was safe, simply because so many of them were also exposed to the ham. Stratification separates the two foods. By this we mean that the risk ratio for one food is computed separately among people who did and did not eat the other, so that within each group the second food is held constant and cannot account for any difference that remains. The food whose risk ratio holds within the strata of the other is the likely source, and the food whose risk ratio falls to 1 within strata looked risky only because of the overlap. Stratification of this kind is the basis of Mantel-Haenszel adjustment for confounding. Worked Example 2.1 applies each step to a small line list from a church supper, with the counts set out in Table 2.2; watch in particular for step 4, where the overlap between the two foods is checked. The knowledge check at the end of this section applies the same steps to a potluck outbreak.

Worked Example 2.1: Two Suspect Foods at a Church Supper

Of 120 guests at a church supper, 36 became ill. Two foods are suspected: ham and potato salad. Table 2.2 gives the counts from the line list.

Table 2.2. Attack rates and risk ratios for ham and potato salad among the 120 guests at a church supper.

FoodAte: totalAte: illAte: attack rateDid not eat: totalDid not eat: illDid not eat: attack rateRisk ratio
Ham803232 / 80 = 40%4044 / 40 = 10%40 / 10 = 4.0
Potato salad602121 / 60 = 35%601515 / 60 = 25%35 / 25 = 1.4
  1. Attack rate for each group: the number ill divided by the number in the group, for those who ate the food and for those who did not.
  2. Risk ratio for each food: the attack rate among those who ate it divided by the attack rate among those who did not. A risk ratio near 1 means no association.
  3. Leading suspect: the ham has the larger risk ratio (4.0 against 1.4), and most of the cases ate it (32 of the 36).
  4. Check the overlap. Fifty guests ate both foods, so the potato salad may look risky only because its eaters also ate the ham. Divide the guests into four groups: both foods (50 guests, 20 ill, 40%), ham only (30 guests, 12 ill, 40%), potato salad only (10 guests, 1 ill, 10%), and neither (30 guests, 3 ill, 10%).
  5. Stratify. Among ham eaters, the potato salad risk ratio is 40% / 40% = 1.0; among guests who did not eat ham, it is 10% / 10% = 1.0. Among potato salad eaters, the ham risk ratio is 40% / 10% = 4.0; among guests who did not eat potato salad, it is also 40% / 10% = 4.0.

The ham's risk ratio holds in both strata, while the potato salad's falls to 1 within strata. The crude association of the potato salad was produced by its overlap with the ham (confounding by the other food), and the ham is the likely vehicle.

Worked Example 2.1 showed that a crude risk ratio can make an innocent food look guilty, and that stratifying on the other food reveals the true vehicle. That analysis settles the question for the guests who are on the line list, but the line list is itself a product of surveillance and case finding, and it inherits the selection described in Section 1. Surveillance systems do not see all populations equally. Data quality, case ascertainment, and willingness to be tested all vary by social position, so outbreak investigators are increasingly expected to ask whose communities are over- or under-represented in the line list and how to adjust the response accordingly. The COVID-19 pandemic made this question impossible to ignore in Canada: differential burdens by neighbourhood income, racialized status, and Indigenous identity were visible in surveillance data once collected, and absent when not.

In the romaine outbreak, a list of healthy people from the same population could not be assembled quickly, so in the United States the cases' eating habits were compared with a survey of healthy people. The same problem arises whenever a population at risk cannot be counted in full: the investigation has to draw a sample. Section 3 sets out how samples are drawn, and Sections 4 and 5 explain how they are analysed and how large they need to be. The key takeaways and knowledge check that follow review the main points of this section.

Key Takeaways

  • 3.1 Cluster, outbreak, epidemic, and pandemic are often used loosely, but each has a specific meaning. A cluster is a grouping of cases in time or space that may or may not be unusual; an outbreak is a cluster that the public-health authority judges to exceed the expected baseline; an epidemic is a large outbreak, often across several jurisdictions; and a pandemic is an epidemic that has spread worldwide. Declaring an outbreak is partly statistical, because current counts are compared with a baseline such as the five-year mean for the same week, and partly operational, because a declaration mobilises resources, triggers a coordinated response, and may require public communication. In a scenario, look first for the baseline and for who has made the judgement: the same number of cases can be ordinary across Canada in a week and an outbreak at one wedding, and a grouping of cases remains a cluster until the authority judges that it exceeds the baseline. Avoid the common trap of reading “pandemic” as “severe”; it is a claim about geography, not severity.
  • 3.2 The CDC 10-step framework gives an investigation its structure: prepare for fieldwork, establish that an outbreak exists, verify the diagnosis, construct a working case definition, find cases and build a line list, carry out descriptive epidemiology by person, place, and time, develop hypotheses, test them with an analytic study (a retrospective cohort when everyone exposed can be listed, a case-control study when they cannot), refine them with further work such as environmental sampling, traceback, and laboratory typing, and finally put control and prevention in place and communicate the findings. A working case definition always has three parts: clinical criteria; restrictions by person, place, and time; and a level of certainty (suspect, probable, or confirmed). When you check a draft definition, look for all three parts, and keep the suspected exposure out of it, because the analytic study must compare people who did and did not have that exposure. The steps look orderly on paper, but in practice several run at once and earlier ones are revisited as information arrives: communication runs throughout, the case definition is revised as the investigation evolves, and control measures often begin before the analytic study is finished, because precaution does not wait for a statistically significant result.
  • 3.3 An epidemic curve is a histogram of cases by date of symptom onset, and its shape points to how people were exposed. A sharp rise to a single peak that falls away within about one incubation period suggests a point source (one brief shared exposure, such as a single meal); a broad plateau suggests a continuous common source that is still active; and successive peaks about one incubation period apart suggest propagated, person-to-person spread. To estimate the exposure window for a point source, choose a time unit shorter than one incubation period, count back from the peak by the average incubation period, and then check the edges: the first case should fall no earlier than the minimum incubation period after the suspected exposure, and the last case of the first wave no later than the maximum. For example, with an average incubation of 2 days and a range of 1 to 5 days, a church supper on Sunday fits a curve that begins on Monday, peaks on Tuesday, and ends on Thursday. Two errors are common: counting back from the peak by the maximum incubation period instead of the average, and reading onsets spread over several days as person-to-person spread, when that spread simply reflects the range of incubation periods; a later second peak among contacts suggests secondary spread but does not by itself rule out a point source for the first wave.
  • 3.4 For foodborne outbreaks that cross jurisdictions, the Foodborne Illness Outbreak Response Protocol (FIORP) sets out who does what among the three federal partners and the provincial and territorial public-health authorities. PHAC leads epidemiology and surveillance, including PulseNet Canada, the laboratory network that sequences the genomes of bacterial isolates; the Canadian Food Inspection Agency (CFIA) leads food-safety investigation, traceback, and product recalls; and Health Canada leads the risk assessment of contaminated products and health-impact guidance. FIORP also defines when an outbreak counts as multi-jurisdictional, sets up an Outbreak Investigation Coordinating Committee for inter-provincial events, and lays out how the partners communicate; the 2008 Maple Leaf listeriosis outbreak drove its modern revisions. In a scenario, match each task to its partner: sequencing isolates and linking cases is PHAC, tracing a product back and recalling it is CFIA, and judging the health risk of a product is Health Canada. A common slip is to give the recall to Health Canada or the sequencing to CFIA.
  • 3.5 The 2017–18 romaine lettuce outbreak of E. coli O157:H7 (42 confirmed cases in five eastern provinces and 25 cases in 15 US states) shows the ten steps and the FIORP partners at work. Laboratory subtyping through PulseNet Canada, with whole-genome sequencing showing that the Canadian and US strains were closely related, established the outbreak and verified the diagnosis, turning cases that looked like scattered background into one investigable cluster. Onsets spread over several weeks, the pattern of a continuous common source. Most Canadian cases reported eating romaine lettuce, but in the United States 55% of cases against 46% of healthy people in a survey was not a significant difference, so only leafy greens could be implicated there; when healthy comparison people cannot be listed quickly, a comparison with survey data or a case-case design trades some validity for considerable speed. PHAC advised people in the affected provinces on 21 December 2017 to consider other lettuce, and the cause of the contamination was never found. When a new scenario describes a few related cases spread across provinces, each below its own baseline, apply the same lessons: rely on genomic linkage rather than provincial counts to see the cluster, expect a case-case or case-control comparison, and recognise that precautionary action before confirmation is a defensible choice.
  • 4.1 A line list, with one row per person, gives you what you need to find the vehicle of a foodborne outbreak. For each food, the attack rate is the number who became ill divided by the number in the group, calculated separately for people who ate the food and people who did not, and the risk ratio is the attack rate among those who ate it divided by the attack rate among those who did not; a risk ratio near 1 means no association. The leading suspect is the food with the largest risk ratio that most of the cases ate: at the church supper, the ham had a risk ratio of 40% ÷ 10% = 4.0 and was eaten by 32 of the 36 cases. Two traps are common: dividing the numbers ill without first dividing by the group sizes, and choosing a food that nearly every case ate when nearly everyone else ate it too, so that its risk ratio is about 1. When two foods were often eaten together, stratify: compute each food's risk ratio separately among people who did and did not eat the other, comparing only groups that differ on one food (never the group that ate both with the group that ate neither); the food whose risk ratio holds in both strata is the likely vehicle, and the one whose risk ratio falls to about 1 was associated with illness only through the overlap.
Knowledge Check: this section

Question 1: A health region usually records three to five cases of shigellosis a month. In the past two weeks, nine cases have been reported, six of them in children who attend the same daycare. The medical health officer has not yet reviewed the situation. Which statement describes it correctly?

An epidemic is a large, often multi-jurisdictional outbreak. A group of cases in one daycare that has not yet been reviewed is a cluster.
A cluster is a grouping of cases that may or may not be unusual; it becomes an outbreak when the public-health authority judges that it exceeds the baseline, using statistical and operational considerations.
Exceeding the usual count is the starting point, but the outbreak threshold is set by the authority's judgement, not by the data alone, because a declaration mobilises resources and communication.
A cluster is simply a grouping of cases in time or space. Sequencing can link cases, but it is not required before a grouping can be called a cluster.
Correct answer: it is a cluster, which becomes an outbreak when the authority judges that it exceeds the baseline. The information needed is the baseline (three to five cases a month), the current count and its grouping in one daycare, and the fact that no judgement has yet been made. A cluster is a grouping of cases in time or space that may or may not be unusual; an outbreak is a cluster that the public-health authority judges to exceed the expected baseline, using a statistical comparison together with operational considerations, because a declaration mobilises resources and public communication. The third option is tempting because "more cases than expected" is the textbook starting point, but the threshold is set by the authority, not by the data alone; an epidemic is a large, often multi-jurisdictional outbreak, and a pandemic is defined by worldwide spread, not by severity. (Section 2, 3.1 Cluster, Outbreak, Epidemic and Pandemic, including the callout of four working terms; Objective 3.1)

Question 2: At Step 4 of an investigation at a ski lodge, the team drafts this case definition: "A case is a guest or staff member of Pine Ridge Lodge whose vomiting or diarrhoea began between 3 and 10 February." Which change would make it a complete working case definition?

The suspected exposure should never be written into a case definition, because the analytic study must compare people who did and did not eat the suspect food.
The time restriction is a required part of a working case definition. Removing the dates would count illness unrelated to this outbreak rather than completing the definition.
The draft has clinical criteria and person, place and time restrictions, but it lacks the third part, levels of certainty, so tested and untested cases cannot be counted consistently.
A working case definition needs clinical criteria, person, place and time restrictions, and levels of certainty; adding confirmed and probable categories supplies the missing third part.
Correct answer: add levels of certainty. A working case definition has three parts: clinical criteria (here, vomiting or diarrhoea), restrictions on person, place, and time (guests or staff of Pine Ridge Lodge, with onset between 3 and 10 February), and a level of certainty (suspect, probable, confirmed), so the draft is missing the third part. The suspected exposure should never be written into the case definition, because the analytic study in Step 8 has to compare people who did and did not eat the suspect food; building the hypothesis into the case count would make that comparison impossible. The third option is the most tempting, since the draft looks complete, but without levels of certainty, tested and untested cases cannot be counted and reported consistently. (Section 2, 3.2 The CDC 10-Step Investigation Framework, the paragraph on the case definition and the Step 4 panel; Objective 3.2)

Question 3: Children at a summer day camp shared two meals: a picnic on Friday 30 May and a barbecue on Monday 2 June. The suspected organism has an average incubation period of 3 days, with a range of 1 to 6 days. The table gives the number of children by date of onset of illness. Which conclusion is best supported?

Date of onsetMon 2 JunTue 3 JunWed 4 JunThu 5 JunFri 6 JunSat 7 JunSun 8 JunMon 9 Jun
Cases014117310
The exposure window is found by counting back the average incubation period from the peak, not the maximum. The picnic also fails the edge check, since the last case began 9 days after it.
Onsets spread over several days reflect the range of incubation periods. Propagated spread would show successive peaks about one incubation period apart, not a single sharp peak.
A single sharp peak indicates a point source; counting back the 3-day average from the Thursday peak gives Monday, and the first and last cases fall 1 and 6 days after the barbecue, within the range.
Counting forward from the first case is a common error. The peak, not the first case, is matched to the average incubation period, and the picnic's last case falls outside the 6-day maximum.
Correct answer: the barbecue on Monday 2 June. The information needed is the shape of the curve, the date of the peak, the dates of the first and last onsets, and the average and range of the incubation period. The curve rises sharply to one peak and falls away within about one incubation period, the shape of a point source; counting back the average incubation period (3 days) from the peak on Thursday 5 June gives Monday 2 June; and the edges fit, because the first case (Tuesday) began 1 day after the barbecue (the minimum incubation) and the last case (Sunday) 6 days after it (the maximum). The picnic fails the edge check, because the last case began 9 days after it, longer than the maximum of 6 days. Counting back from the peak by the maximum incubation period, or counting forward from the first case, are the two common errors; and onsets spread over several days reflect the range of incubation periods, not person-to-person spread, which would show successive peaks about one incubation period apart. (Section 2, 3.3 Reading an Epidemic Curve, the paragraph on estimating the exposure window; Objective 3.3)

Question 4: Over six weeks, PulseNet Canada receives Listeria monocytogenes isolates from seven people in three provinces. No province has more cases than its usual baseline. Under FIORP, and as in the 2017–18 romaine lettuce outbreak, which description of the response is correct?

Under FIORP, PHAC leads epidemiology and surveillance including PulseNet Canada sequencing, CFIA leads traceback and recalls, and Health Canada assesses risk; sequencing turns scattered cases into one cluster.
Genomic surveillance exists to catch exactly this: cases that look like background in each province but are closely related, as in the romaine outbreak across five provinces.
This swaps the roles. PHAC runs PulseNet Canada sequencing, CFIA leads traceback and recalls, and Health Canada leads the risk assessment.
FIORP coordinates the partners when cases span provinces. Waiting until a food is confirmed would defeat the purpose of linking scattered cases by sequencing early.
Correct answer: PHAC links the cases by whole-genome sequencing, CFIA traces the food and leads any recall, and Health Canada assesses the risk. The information needed is the role of each FIORP partner and what genomic surveillance added in the romaine outbreak. PHAC leads epidemiology and surveillance, including PulseNet Canada, the laboratory network that sequences bacterial isolates; CFIA leads food-safety investigation, traceback, and recalls; and Health Canada leads risk assessment. In the romaine outbreak, laboratory subtyping, confirmed by sequencing, showed that isolates from five provinces were closely related, turning cases that looked like background into one investigable cluster. The second option repeats the error that genomic surveillance exists to prevent, and the third swaps the roles, a common slip. (Section 2, 3.4 The Canadian Foodborne Illness Outbreak Response Protocol and 3.5 Case Study: The 2017–18 Romaine Lettuce Outbreak; Objectives 3.4 and 3.5)

Question 5: Of 200 people at a hockey tournament banquet, 40 became ill. The investigators ran a retrospective cohort study of everyone on the guest list, with the results below. Which food is the most likely vehicle, and what is its risk ratio?

FoodAte: totalAte: illDid not eat: totalDid not eat: ill
Chicken skewers12036804
Rice salad1002510015
Fruit platter19038102
Most cases ate the fruit platter because nearly everyone did. Its attack rates are 38 ÷ 190 = 20% and 2 ÷ 10 = 20%, a risk ratio of 1.0, which shows no association.
Attack rates of 36 ÷ 120 = 30% among chicken eaters and 4 ÷ 80 = 5% among non-eaters give a risk ratio of 6.0, by far the largest, and most cases ate the chicken.
The rice salad's attack rates are 25% and 15%, a risk ratio of about 1.7, which is much weaker than the chicken's 6.0.
9.0 divides the numbers ill (36 ÷ 4) without dividing by group size. The groups differ (120 against 80), so attack rates must be compared: 30% ÷ 5% = 6.0.
Correct answer: the chicken skewers, with a risk ratio of 6.0. For each food, divide the number ill by the number in the group to get the two attack rates, then divide the attack rate among those who ate the food by the attack rate among those who did not. Chicken skewers: 36 ÷ 120 = 30% and 4 ÷ 80 = 5%, so the risk ratio is 30 ÷ 5 = 6.0; rice salad: 25 ÷ 100 = 25% and 15 ÷ 100 = 15%, a risk ratio of about 1.7; fruit platter: 38 ÷ 190 = 20% and 2 ÷ 10 = 20%, a risk ratio of 1.0. The chicken has by far the largest risk ratio and was eaten by most of the cases (36 of 40). The fruit platter is tempting because even more cases ate it, but nearly everyone ate it, ill or not, and its risk ratio of 1.0 shows no association; 9.0 comes from dividing the numbers ill (36 ÷ 4) without first dividing by the group sizes, which differ (120 against 80). (Section 2, 4.1 From Line List to Risk Ratio and Worked Example 2.1; Objective 4.1)

Question 6: At the same banquet, the rice salad was served beside the chicken skewers, and many guests took both. The table divides the 200 guests by whether they ate each food. What does a stratified analysis of these groups show?

GroupAte chicken skewersAte rice saladGuestsIll
1YesYes8024
2YesNo4012
3NoYes201
4NoNo603
Crude risk ratios above 1 can arise from overlap between foods. Stratifying shows the rice salad's risk ratio is 1.0 within each chicken stratum, so it is not an independent source.
This reverses the confounding. The chicken's risk ratio stays at 6.0 whether or not rice salad was eaten, while the rice salad's falls to 1.0 within strata of chicken.
Groups 1 and 4 differ on both foods at once, so comparing them cannot separate the two. Within strata of chicken, the rice salad's risk ratio is 1.0.
Holding chicken constant, rice salad's attack rates are 30% against 30% and 5% against 5%, a risk ratio of 1.0; the chicken's stays at 6.0 in both rice strata, so the chicken is the likely vehicle.
Correct answer: the rice salad's association came from its overlap with the chicken. First, compute the attack rate in each group: 24 ÷ 80 = 30%, 12 ÷ 40 = 30%, 1 ÷ 20 = 5%, and 3 ÷ 60 = 5%. Then hold one food constant and compare the other: among chicken eaters, the rice salad risk ratio is group 1 against group 2, 30% ÷ 30% = 1.0, and among guests who did not eat chicken it is group 3 against group 4, 5% ÷ 5% = 1.0; the chicken risk ratio is 30% ÷ 5% = 6.0 among rice salad eaters (group 1 against group 3) and also 6.0 among guests who skipped it (group 2 against group 4). The food whose risk ratio holds within strata of the other is the likely vehicle, and the food whose risk ratio falls to about 1 was associated with illness only through the overlap (confounding by the other food). Comparing group 1 with group 4 is the tempting error, because those guests differ on both foods at once, so the comparison cannot separate them. (Section 2, 4.1 From Line List to Risk Ratio, Worked Example 2.1, steps 4 and 5; Objective 4.1)

Question 7: A community potluck dinner was held from 6 to 8 p.m. on a Saturday, and 23 of the 140 people who attended developed vomiting and diarrhoea. The suspected organism has an average incubation period of about 24 hours, with a range of 6 to 72 hours. The first guest became ill at 3 a.m. on Sunday, onsets rose to a single sharp peak on Sunday evening, and the last guest became ill at 6 p.m. on Monday. Is the epidemic curve consistent with a single exposure at the potluck?

Onsets spread over a day or two reflect the range of incubation periods. Propagated spread would show successive peaks, not one sharp peak.
The exposure window is found by counting back the average incubation period from the peak, not the maximum. Counting back 24 hours from the peak lands on Saturday evening.
A single sharp peak fits a point source; counting back the 24-hour average from the peak points to Saturday evening, and the first and last onsets fall within the 6 to 72 hour range.
The number of cases does not show a single exposure. The argument rests on the shape of the curve and on matching its peak and edges to the incubation period.
Correct answer: yes, the curve fits a single exposure at the potluck. The information needed is the shape of the curve and the incubation period. A single sharp peak is the point-source shape; counting back one average incubation period (about 24 hours) from the Sunday-evening peak lands on Saturday evening; and the edges fit, since the first case began 7 to 9 hours after the meal and the last 46 to 48 hours after it, inside the range of 6 to 72 hours. Because the incubation period is measured in hours, the curve should be drawn in blocks of a few hours so that its shape is not smoothed away. The first option is tempting, but onsets spread over a day or two reflect the range of incubation periods, not person-to-person spread, and the second option counts back by the maximum instead of the average incubation period. (Section 2, 3.3 Reading an Epidemic Curve, Figure 2.1 and the paragraph on estimating the exposure window; Objective 3.3)

Question 8: At the same potluck, of the 80 guests who ate the chicken curry, 20 became ill, and of the 60 who did not, 3 became ill. Of the 80 who ate the naan bread, 16 became ill, and of the 60 who did not, 7 became ill. Many guests ate both: 60 ate both foods (15 ill), 20 ate only the curry (5 ill), 20 ate only the naan (1 ill), and 40 ate neither (2 ill). Which statement is correct?

Curry gives 25% ÷ 5% = 5.0 and naan about 1.7, but within strata of curry the naan's attack rates are 25% against 25% and 5% against 5%, so the naan's association came from overlap.
Crude risk ratios mix the two foods, because 60 of the 80 naan eaters also ate curry. Stratifying shows the naan's risk ratio is 1.0, so it is not an independent vehicle.
These divide the number ill in each food group by the number of cases. An attack rate uses everyone in the group as its denominator, for example 20 ÷ 80 for curry eaters.
Within strata of curry the naan's risk ratio falls to 1.0, not 5.0. It is the curry's risk ratio that stays at 5.0 in both strata of naan.
Correct answer: the naan's association is produced by its overlap with the curry, which is the likely vehicle. The information needed is an attack rate for every group, using everyone in the group as the denominator. Curry: 20 ÷ 80 = 25.0% against 3 ÷ 60 = 5.0%, a risk ratio of 5.0. Naan: 16 ÷ 80 = 20.0% against 7 ÷ 60 = 11.7%, a risk ratio of about 1.7. Stratifying on curry, the naan attack rates are 15 ÷ 60 = 25% (both foods) against 5 ÷ 20 = 25% (curry only) among curry eaters, and 1 ÷ 20 = 5% (naan only) against 2 ÷ 40 = 5% (neither) among the others, so the naan risk ratio is 1.0 in both strata, while the curry risk ratio stays at 5.0 in both strata of naan. The second option stops at the crude risk ratios, which mix the two foods because 60 of the 80 naan eaters also ate the curry, and the third divides by the number of cases instead of the number of people in each group. (Section 2, 4.1 From Line List to Risk Ratio and Worked Example 2.1; Objective 4.1)
Section 3 of 5

Drawing a Sample

⏱ Estimated reading time: 35 minutes

Section 3 of 5

Drawing a Sample

Census and sample, populations and frames, and the probability and non-probability designs.

First distinction

Census vs. sample

Census

Every individual measured. Only measurement error applies.

Sample

A subset measured. Both measurement error and sampling error apply, but a well-designed sample provides nearly the same information at a fraction of the cost.

Canada runs both: the Census of Population (every 5 years) and surveys like the CCHS and CHMS that produce population estimates with sampling error attached.

Three levels

The population hierarchy

Study Sample Source Population Target Population Internal validity External validity

The sampling frame is the list of units in the source population. In Canadian practice it is often the Statistics Canada Address Register, an LFS area frame, or a provincial health insurance registry.

The defining property

What makes a sample a probability sample

Every element in the sampling frame has a known, non-zero probability of selection, assigned through a formal random process.

This is what separates probability sampling from all non-probability designs. The known probability is what allows you to compute unbiased estimates and valid standard errors.

Random does not mean haphazard. A formal, reproducible process is required: computer-generated numbers or a random number table.

The building blocks

Simple random and systematic sampling

Simple random

Equal probability for all. Needs complete list. All standard analyses apply directly.

Systematic

Pick a random start, then every jth unit. Practical for sequential records. Watch for periodicity.

Sampling interval
\[ \color{#0B7B6B}{j} = \frac{\color{#C2410C}{N}}{\color{#6D28D9}{n}} \]
j sampling interval (take every j-th unit)N population sizen desired sample size
Dividing first

Stratified random sampling

Source population Stratum A Stratum B Stratum C Stratum D SRS or systematic within each stratum; Neyman allocation for optimal precision
Sampling groups

Cluster sampling

Primary sampling unit = the cluster (household, class, clinic).

Randomly select clusters; measure everyone inside each selected cluster.

Advantages: only a cluster list is needed; fewer locations to visit.

Limitation: within-cluster similarity inflates variance relative to SRS for the same n.

Selected Cluster Cluster Cluster
Two-stage selection

Multistage sampling

Select primary sampling units (PSUs), then randomly sample individuals within each PSU rather than taking everyone.

CCHS design

Stratify by health region → cluster by dwelling within each region → select one person per dwelling at random.

Key requirement

Equal selection probabilities require probability-proportional-to-size (PPS) at the PSU stage with a fixed number per PSU, or equal-probability PSU selection with a constant sampling fraction within each PSU.

Carry forward

Targeted sampling and the design decision

Targeted (risk-based) sampling focuses on high-risk strata: efficient for rare diseases but requires prior knowledge of risk factors and cannot yield standard risk ratios without external data.

Decision question

Does your frame support random selection? If yes, choose a probability design. If no, document the non-probability design and its analytic limits.

Complexity cost

Stratification, clustering, and multistage designs require design-corrected analysis in Section 4. The analysis must match the design.

No random selection

Three non-probability designs

Judgement

Investigator selects participants believed to be representative. Subject to unmeasured selection bias.

Convenience

Whoever is easiest to reach. Fast and cheap; rarely generalisable.

Purposive

Deliberate targeting of specific individuals or groups. Common in qualitative research and case series.

Hidden populations

Snowball and hybrid designs

Snowball sampling (Goodman, 1961): participants recruit peers. No sampling frame needed. Used for hidden populations.

Respondent-driven sampling

Heckathorn (1997). Uses structured referral chains to estimate population parameters. Assumes equilibrium across chains.

Time-location sampling

Systematic sampling at known venues. Used in HIV behavioural surveillance. Requires venue attendance to be unrelated to the outcome.

Carry forward

What to take into the next section

  • The sampling frame decides who can be selected; people missing from the frame cannot be sampled.
  • Probability designs give every unit a known, non-zero chance of selection.
  • Non-probability samples cannot support prevalence estimates, because each participant's selection probability is unknown.
  • Selection that depends on the outcome is the formal criterion for selection bias (Lesson 7).

Overview

When a population cannot be measured in full, a study draws a sample, and the way the sample is drawn determines much of what the results can show. This section distinguishes a census from a sample and the errors each carries, describes the nested populations that a study touches, the sampling frame that links them and the two kinds of validity that attach to them, and works through the probability sampling designs. It ends with the non-probability designs used when no sampling frame exists and the bias they introduce.

Learning Objectives

  • 5.1 Distinguish a census from a sample and the sources of error that each involves.
  • 5.2 Define the target population, source population, and study sample for a described study, name a suitable sampling frame and a group it would miss, and distinguish internal from external validity.
  • 6.1 Define a probability sample, describe simple random, systematic, stratified, cluster, multistage, and targeted sampling with the advantages and limitations of each, and name the design used in a described study.
  • 6.2 Describe judgement, convenience, purposive, and chain-referral samples, explain why non-probability samples are generally unsuitable for estimating prevalence, and predict the direction in which a non-probability sample would bias an estimate.

5.1 Census and Sample

When we conduct research, we need data from either all individuals in a population or a subset of them. The process of obtaining this data is called measurement. In a census, every individual in the population is evaluated. In a sample, data are collected from only a subset. Sampling is generally more convenient and less costly than conducting a full census. Interestingly, even a census can be viewed as a kind of sample: it captures the population at one point in time, making it a "sample" of the population over time.

The two approaches differ in the errors they carry, and the distinction is worth committing to memory.

Key distinction: sources of error in a census and a sample

In a census, the only source of error is the measurement itself. With a sample, you contend with both measurement error and sampling error. However, a well-planned sample can provide virtually the same information as a census at a fraction of the cost.

The two kinds of error arise at different points. Measurement error is a value misread or recorded wrongly, as when a blood-pressure cuff is the wrong size or a respondent misremembers how often they smoke. It happens at the moment each person is measured, so it can affect a census just as much as a sample. Sampling error, by contrast, is the chance difference between the people who happened to be selected and the whole population. For example, if 20% of adults in a city smoke, a random sample of 200 adults might, purely by chance, contain 18% or 23% smokers, even if every answer were recorded perfectly. A census has no sampling error because no one is left out, but it still carries measurement error. Random selection, the subject of 6.1, controls sampling error only, so neither a census nor a random sample is free of measurement error.

Canada runs both kinds of data collection at the population scale, and you will encounter all of them in public health practice. The Census of Population (Statistics Canada, every 5 years; most recent 2021) is a near-complete enumeration of every household in Canada. The short-form census goes to all households, and the long-form census goes to a 25% mandatory sample. The census provides the denominators behind almost every population health rate you will calculate. The Canadian Community Health Survey (CCHS), run by Statistics Canada, Health Canada, and PHAC, is a continuous cross-sectional sample survey (tens of thousands of respondents a year; about 100,000 dwellings were selected from the area frame for 2025) covering self-reported health, behaviours, and health-care use; it is the flagship descriptive survey for population health surveillance. The Canadian Health Measures Survey (CHMS), from the same partners, is a multi-stage sample survey that adds direct physical measurements (blood pressure, biomarkers, fitness) and a biobank to self-report data; it is smaller (~5,700 respondents per cycle) but anchors objective measurement of population health. The National Population Health Survey (NPHS) was the longitudinal predecessor (1994–2011) to the CCHS and is still used for life-course research. The Census gives you a denominator and demographic context, while the CCHS and CHMS give you population estimates of health states with sampling error attached. Choosing among them is the first applied sampling decision a public-health analyst makes.

A sample is drawn from a defined population, and a study involves several nested populations. Subsection 5.2 sets out how they relate to one another.

5.2 Populations, Sampling Frames and Validity

5.2.1 The Hierarchy of Populations

Understanding the different populations involved in a study is essential for evaluating validity. There are three key populations to consider, each nested inside the next, and Figure 2.2 places the study sample inside the source population and the source population inside the target population. The labels next to the arrows (external validity, sampling frame, internal validity) are the technical vocabulary you will use to talk about how a sample inherits or loses information from the broader population it was meant to represent.

Target Population The broadest group to which you want to generalize results External validity Source Population The population from which study subjects are drawn Sampling frame Study Sample The individuals who actually participate and provide data Internal validity

Figure 2.2. Hierarchy of populations in epidemiologic research. The target population is the broadest; the source population is the accessible subset; the study sample consists of those who actually participate.

Each layer has a precise meaning. The target population is the group the results are meant to describe, the source population is the listable group from which participants are actually drawn, and the study sample is the people who end up providing usable data. Expand each panel below for a fuller definition of each layer, with examples from a study of rainwater cisterns and diarrhea in Brazil (Marcynuk et al., 2013).

Target Population ▼

The target population is the population to which you want to extrapolate your results. It is often not clearly defined and may vary depending on the perspective of the person interpreting the study. For example, researchers studying rainwater cisterns in Pernambuco State, Brazil might define the target as that state, while someone else may want to generalize the findings to all semi-arid regions of Brazil.

Source Population ▼

The source population is the population from which study subjects are actually drawn. All units in the source population should be "listable" and have a non-zero probability of being included in the study. For example, in a diarrhea study in Brazil, the source population included families from households participating in the One Million Cisterns Project (OMCP).

Study Sample ▼

The study sample (or study group) consists of the individuals who actually end up in the study. It is typically a subset drawn from the source population. Researchers determine the necessary sample size, draw their sample, collect data from eligible subjects, and the final study sample consists of those who agreed to participate and whose data met quality requirements.

The Brazil examples show that the target population can shift with the person interpreting the study, while the source population must be something that can be listed.

Two kinds of validity attach to this hierarchy. Internal validity concerns whether the results are correct for the source population, and external validity concerns whether they can be generalized from the source population to the target population. Internal validity is threatened by selection, information, and confounding biases.

Applying the hierarchy to a described study means writing one sentence for each layer. Suppose a health authority wants to know how common food insecurity is among post-secondary students in British Columbia this year. The target population is all post-secondary students in the province this year, the group the estimate is meant to describe. The source population is the students enrolled at the institutions that agree to share their enrolment lists, the group that can be listed and sampled. The study sample is the students who are selected, consent, and complete the survey. The sampling frame is the set of enrolment lists, and it misses students at institutions that decline to take part and students who enrol after the lists are drawn. Internal validity asks whether the estimate is correct for the enrolled students on the lists; external validity asks whether it can be generalised to all students in the province, including those the frame misses. A gap in the frame therefore threatens external validity, while non-response among students who were on the lists affects how well the study sample represents the source population. The term study population is used in two ways in the literature: some texts, and the glossary of this lesson, use it for the source population, while others, including the course examinations, use it for the study sample. When you use the term, say which layer you mean.

5.2.2 The Sampling Frame

The example introduced the frame as the link between the source population and the sample. Formally, the sampling frame is the list of all sampling units in the source population. Sampling units are the basic elements that will be sampled (e.g., households, individuals). A complete list of all sampling units is required for drawing a simple random sample, though some other methods do not require such a complete listing. In the Brazil study of water cisterns and diarrhea, a suitable sampling frame was the list of all households eligible for the One Million Cisterns Project. Once households were selected, a separate strategy was used for selecting individuals within each household.

National-scale Canadian surveys rarely have a single tidy list of every person. Instead, they assemble a frame from administrative listings. The Statistics Canada Address Register (AR) is the dwelling-level frame used by the Census and many StatCan household surveys. The Labour Force Survey (LFS) area frame, which is itself based on Census enumeration areas, supplies part of the CCHS sample. Provincial health insurance registries (e.g., the BC Medical Services Plan client registry) are close to a population census of residents and the backbone of administrative-data research at Population Data BC (PopData BC). Disease registries such as the Canadian Cancer Registry or provincial reportable-disease lists serve as case frames for surveillance. Each frame has different coverage: a registry-based frame misses people without provincial coverage, and an LFS area frame excludes residents on First Nations reserves and in institutions. The frame, not the questionnaire, is usually where exclusions and selection bias enter, and naming the group a frame misses is the first step in judging external validity.

Once a frame exists, units can be selected from it by a formal random process. Subsection 6.1 defines the probability sample that such a process produces and the designs that produce it.

6.1 Probability Sampling Designs

6.1.1 What Makes a Sample a Probability Sample

A probability sample is one in which every element in the population has a known, non-zero probability of being included. By this we mean two things. Non-zero means that no one on the frame is ruled out in advance: everyone has some chance of being chosen. Known means that the investigator can say exactly what that chance is, for example 1 in 13 for every patient in a list of 13,000 from which 1,000 are drawn. The chances do not have to be equal, but they do have to be calculable. This implies that a formal process of random selection has been applied to the sampling frame, since only a defined random process produces probabilities that can be written down. The key advantage is that probability samples allow for valid statistical inferences about the source population. Because the chance of selection is known, the analyst can work out how far a sample estimate is likely to stray from the true population value, and can correct for any groups that were deliberately sampled more or less heavily than others. Random selection uses a formal, reproducible process (e.g., computer-generated random numbers, random number tables); it is not the same as selecting participants haphazardly or arbitrarily, and a sample picked "at random" by an interviewer on a street corner is haphazard, not random.

6.1.2 The Six Probability Designs

A formal random process can be applied to a sampling frame in several ways, and the designs differ in whether they need a complete list of the source population and in how units are selected from it. Simple random and systematic sampling select individuals from a list. Stratified sampling divides the list into groups and samples from every group, and targeted sampling builds on stratification by concentrating on the groups where disease is most likely. Cluster and multistage sampling select groups, and then study everyone in each chosen group or a random sample within it. Open each tab below to see how each design works, a worked-through hospital or school example of it, and its main advantage and limitation.

Simple Random Sample

In a simple random sample, every study subject in the source population has an equal probability of being included. A complete list of the source population is required, and a formal random process is used to select individuals.

Example: To study wait times in a hospital emergency room, you need 1,000 records from 13,000 admissions over the past year. You randomly generate 1,000 numbers between 1 and 13,000 and pull those records.

Advantage: Conceptually simple; all standard statistical analyses apply directly.

Limitation: Requires a complete list of the entire source population.

Systematic Random Sample

In a systematic random sample, a complete list is not required; you only need an estimate of the total population and sequential access to individuals. The sampling interval (j) is computed as the population size divided by the desired sample size.

How it works: Randomly pick a starting point between 1 and j, then select every jth subject after that.

Example: To sample 1,000 from 13,000 emergency patients, the sampling interval is 13. Randomly pick a number between 1 and 13 for your starting patient, then select every 13th patient thereafter. So if your random start happens to be 7, you would sample patients 7, 20, 33, 46, and so on down the list.

Caution: Bias may occur if the factor you are studying is related to the sampling interval (e.g., periodic patterns in admissions).

Stratified Random Sample

The population is divided into mutually exclusive strata based on factors likely to affect the outcome. Then, within each stratum, a simple or systematic random sample is chosen. The mathematical foundations of stratification, including the now-standard optimum (Neyman) allocation rule for assigning sample size across strata, were laid out by Neyman (1934) in his landmark Royal Statistical Society paper that effectively founded probability sampling theory.

In proportional stratified sampling, the number sampled from each stratum is proportional to that stratum's share of the total population.

Three key advantages:

  • Ensures all strata are represented in the sample.
  • Can produce more precise overall estimates than a simple random sample because between-strata variation is removed.
  • Allows estimation of stratum-specific outcomes.

Example: If hospital wait times differ between males and females, stratify records by sex and randomly sample within each group.

Cluster Sampling

A cluster is a natural grouping of study subjects with one or more common characteristics (e.g., a household is a cluster of people; a classroom is a cluster of students; a clinic is a cluster of patients).

In cluster sampling, the primary sampling unit (PSU) is the cluster itself, and it is often larger than the unit of concern. Every individual within a selected cluster is included in the sample.

Example: To estimate smoking prevalence among Grade 12 students, randomly select 10 of 47 Grade 12 classes and survey all students in those 10 classes.

Advantage: Easier when getting a list of clusters is simpler than listing all individuals. Often cheaper to visit fewer locations.

Limitation: Individuals within a cluster tend to be more alike, increasing sampling variation for a given sample size compared to SRS.

Important: A sample is only a "cluster sample" if the group is the sampling unit and the individuals within it are the unit of concern. If the group itself is the unit of concern (e.g., "does anyone in the household smoke indoors?"), it is not a cluster sample.

Multistage Sampling

Multistage sampling is similar to cluster sampling, except that after selecting primary sampling units (PSUs), a sample of secondary sampling units (individuals) is drawn within each PSU rather than surveying everyone.

Example: To study smoking among students, first randomly select 10 classes (PSUs), then randomly select 5 students from each class rather than surveying all students in every class. Within-household selection in face-to-face surveys is often done using a Kish grid, the objective respondent-selection procedure introduced by Kish (1949).

Equal chances do not happen automatically in a two-stage design, because a person's overall chance of selection is the chance that their PSU is chosen multiplied by the chance that they are chosen within it. If every class had the same chance of being picked and then 5 students were taken from each, a student in a class of 10 would have a 5 in 10 chance at the second stage while a student in a class of 40 would have only a 5 in 40 chance, so students in small classes would be over-represented. To ensure all individuals have the same probability of being selected, either choose PSUs with probability proportional to their size (PPS) and take a fixed number of individuals from each, or choose PSUs with equal probability and use a constant sampling proportion within each PSU. In the first approach, the large class is more likely to be picked but each of its students is less likely to be chosen once it is, and the two stages balance out. In the second, every class has the same chance and the same fraction, such as one student in four, is taken from whichever classes are chosen.

The number of individuals per cluster (ni) can be optimized by balancing within-cluster and between-cluster variance against the costs of sampling groups versus individuals.

Targeted (Risk-Based) Sampling

Targeted sampling stratifies the source population based on characteristics associated with the probability of disease occurrence, then focuses sampling on strata where disease is most likely to be found.

Individuals are assigned point values based on their probability of having the disease of interest, and sampling proceeds until a predetermined number of points have been sampled. By this we mean that a high-risk individual counts for more toward the target than a low-risk one, so that fewer high-risk individuals are needed to reach it. For example, if older animals in a herd, or people with a known exposure, are several times more likely to be infected, each of them might be worth several points, and the investigator can reach the target with a much smaller sample by concentrating on them. This is an unequal probability sampling strategy; some individuals may even have a zero probability of inclusion. Because of this, a targeted sample is well suited to the question of whether a disease is present at all, but it cannot by itself give a prevalence for the whole population.

Advantage: Requires a much smaller sample to detect rare diseases when key risk characteristics can be identified.

Limitation: Key epidemiological parameters (e.g., risk ratios) may not be known for the study population and must be estimated from other evidence.

Stratified and cluster sampling both divide the population into groups, which makes them easy to confuse, but they have opposite effects on precision. The difference lies in which groups end up in the sample. In a stratified design every group is sampled, so the sample is guaranteed to contain the right mix of, say, urban and rural residents, and chance can no longer produce a sample that is too heavily urban. That removes one source of sample-to-sample variation, which is why precision improves. In a cluster design only some groups are sampled, and the people within a group tend to resemble one another. For example, students in the same class share a teacher, a neighbourhood, and friends, so surveying a whole class tells you less than surveying the same number of students scattered across the school. Which classes happen to be chosen then adds a new source of variation, and precision falls. Figure 2.3 contrasts them: a stratified design samples from every group and gains precision, while a cluster design samples only some of the groups and loses it.

Stratified versus cluster sampling: strata are internally similar and all are sampled; clusters are internally similar and only some are sampledStratifiedsample from every stratumeach stratum internally similar;all strata sampled → gain precisionClustersample whole clusterseach cluster internally similar;only some clusters sampled (outlined)→ lose precision (design effect)
Figure 2.3. Stratified versus cluster sampling. Strata are relatively homogeneous internally and every stratum is sampled, which removes between-stratum variation and improves precision. Clusters are relatively homogeneous internally and only a subset is sampled, so cases within a cluster carry overlapping information and precision falls (the design effect).

6.1.3 Naming the Design of a Described Study

The six designs can be compared on whether they need a complete list of the population, on their key advantage, and on their key limitation. Table 2.3 sets out this comparison.

Table 2.3. The six sampling designs compared by whether each requires a complete list, its key advantage, and its key limitation.

MethodRequires Complete List?Key AdvantageKey Limitation
Simple RandomYesSimple; all standard analyses applyNeeds complete population list
SystematicNo (needs estimate)Practical; easy to implementPeriodic bias if factor linked to interval
StratifiedYes (within strata)More precise; ensures representationNeeds to know stratum membership
ClusterList of clusters onlyCheaper; no need to list individualsHigher variance than SRS for same n
MultistageList of PSUs onlyFlexible; cost-effectiveComplex design; needs more subjects
TargetedNo (risk-based)Efficient for rare diseasesNeeds prior knowledge of risk factors

To name the design of a sample that is described to you, ask what is selected at each stage and how. If individuals are drawn at random from one complete list, the design is simple random; if every jth unit is taken after a random start, it is systematic; if the list is first divided into groups and a random sample is drawn from every group, it is stratified; if some groups (schools, classes, clinics, households) are chosen at random and everyone in them is studied, it is a cluster sample; and if some groups are chosen at random and a random sample of individuals is then drawn within each chosen group, it is a two-stage, or multistage, cluster sample. The difference between stratified and cluster sampling is worth holding onto: a stratified design samples from every group, while a cluster design samples only some of the groups, so a design is not stratified just because the population was divided into groups.

A systematic sample needs one calculation, the sampling interval, which is the population size divided by the desired sample size, j = N ÷ n. To take 1,000 records from 13,000 emergency admissions, j = 13,000 ÷ 1,000 = 13, so a random starting number is chosen between 1 and 13 and every 13th record is taken after it; with a random start of 7, the sample is records 7, 20, 33, 46, and so on. Check that the list has no pattern that repeats at the same interval, because a periodic pattern can bias a systematic sample.

Large national surveys combine several of these designs, and naming them stage by stage is the same routine applied more than once. Worked Example 2.2 applies this routine to the Canadian Community Health Survey, which combines stratification, clustering, and selection within households in a single multistage design; watch for what is selected at each stage.

Worked Example 2.2: How the CCHS Combines These Methods

The Canadian Community Health Survey illustrates a real multistage probability design in action:

  1. Stratification: the population is first stratified by health region (in all but two provinces, as of the 2025 cycle), and a target sample size is allocated to each so that every region produces stable estimates.
  2. Clustering: within each health region, dwellings are sampled from the LFS area frame (groups of dwellings that share a geographic boundary). This is the cluster stage.
  3. Selection within cluster: one person is randomly selected from each chosen household to complete the interview.
  4. Additional sample: as of the 2025 cycle, about 28,000 more households are drawn from respondents to the 2021 Census long form; earlier cycles used a telephone frame for this purpose.

The result is a probability sample where every Canadian resident has a known, non-zero chance of selection, but the selection probability differs by region, household size, and frame. That is why CCHS data must be analysed with survey weights and bootstrap replicate weights (covered in Section 4, 7.4).

Worked Example 2.2 showed that, read stage by stage, the CCHS is stratified by health region, clustered within each region, and sampled again within each household. Every one of these probability designs depends on a sampling frame; subsection 6.2 turns to what happens when no frame exists.

6.2 Non-Probability Sampling

6.2.1 Judgement, Convenience, Purposive and Chain-Referral Samples

Samples drawn without an explicit method for determining each individual's probability of selection are known as non-probability samples. Whenever there is no formal process for random selection, the sample should be considered non-probability. A non-probability sample is not automatically wrong, and whether it misleads depends on how people came to be in it. Sample selection that is unrelated to the outcome of interest leaves inference intact. For example, recruiting people by the first letter of their surname is unlikely to have anything to do with their blood pressure. Selection that depends on unmeasured determinants of the outcome is a different matter. If the people who volunteer for a heart-health study are more health-conscious than those who do not, and health-consciousness also lowers blood pressure, the sample will show lower blood pressures than the population has, and nothing in the data will reveal it. This distortion is called selection bias, a form of specification error formalised by Heckman (1979) and reviewed for hidden populations by Sudman & Kalton (1986). There are three main types, and a fourth family is used for hidden populations.

Judgement, convenience, and purposive samples differ in why the participants were chosen: because the investigator judged them representative, because they were easy to reach, or because they had a particular attribute such as an exposure or a disease. Click each card below to see how each type is chosen and where it is used, including the convenience-based selection of comparison households in the Brazil cistern study.

Judgement
Sample
Click to learn more
Convenience
Sample
Click to learn more
Purposive
Sample
Click to learn more

None of the three uses a formal random process applied to a sampling frame, although a purposive sample becomes a probability sample of a defined subset if participants are then drawn at random from all units that meet the study criteria.

Chain-referral (snowball) sampling, first formalised by Goodman (1961), recruits hidden-population members through peer referrals and is widely used when no sampling frame exists. A hidden population is one for which no list exists and whose members may not wish to be identified, such as people who inject drugs or sex workers. The investigator recruits a few initial participants, often called seeds, and asks each of them to refer others they know, who in turn refer others. Because people tend to know people like themselves, and because well-connected people are more likely to be referred, the final sample reflects the social networks of the seeds more than the population as a whole. Two newer hybrids partially recover probability-style inference. By this we mean that they add structure to recruitment and record extra information, such as how many people in the population each participant knows, so that approximate selection probabilities can be estimated and used to weight the results: respondent-driven sampling (RDS), introduced by Heckathorn (1997) and extended with unbiased estimators by Salganik & Heckathorn (2004); and time-location (venue-based) sampling, applied at national scale for HIV behavioural surveillance by MacKellar and colleagues (2007). Magnani, Sabin, Saidel, & Heckathorn (2005) review when each design is appropriate for hard-to-reach populations.

6.2.2 Why Non-Probability Samples Cannot Estimate Prevalence

All four designs share one consequence, because none of them lets anyone say what share of the population each participant represents.

Non-probability samples and prevalence

Non-probability samples are generally inappropriate for descriptive studies because you cannot generalize prevalence estimates to the source population without knowing each individual's probability of being included. However, non-probability methods are commonly used in analytical studies where comparing exposure groups is the priority. The reason is that an analytic study asks whether the outcome differs between exposed and unexposed people within the sample, not how common the outcome is in the whole population. A comparison of this kind can still be valid when the sample is not representative, provided the exposed and unexposed groups were selected in the same way, so that the method of recruitment does not itself create the difference between them.

The bias that a non-probability sample introduces into a descriptive estimate can usually be predicted in direction, even when its size is unknown. Ask how the people who are easiest to reach differ from the target population on the outcome itself. A convenience sample of people sitting in a walk-in clinic waiting room, used to estimate how common influenza-like illness is in a town, over-represents people who are ill or recently ill and people who readily seek care, so it would overestimate the prevalence. A convenience sample recruited at a fitness centre, used to estimate how common physical inactivity is, over-represents active people and would underestimate it. Stating the expected direction, and the reason for it, is part of describing the limitation of any sample that was not drawn by a formal random process from a sampling frame. A complete answer names the design, explains why the selection probabilities are unknown, states the direction of the bias with its reason, and recognises that recruiting more people in the same way would not remove the bias.

This section has distinguished a census from a sample, set out the populations and the sampling frame that a study involves, and described the probability and non-probability designs. Section 4 explains why a probability sample supports inference and how its design enters the analysis. The key takeaways and knowledge check that follow review these points.

Key Takeaways

  • 5.1 A census measures every member of a population, while a sample measures only a subset, and the difference decides which errors each one carries. A census is still subject to measurement error (a question misread, a value recorded wrongly), while a sample has both measurement error and sampling error, the chance difference between the people selected and the whole population. A well-planned sample can give almost the same information as a census at a fraction of the cost, which is why Canada runs the Census of Population for denominators and demographic context alongside sample surveys such as the CCHS and CHMS for health estimates. In a question, do not claim that a census is free of error or that random selection removes measurement error: random selection controls sampling error only.
  • 5.2 Every study involves several nested populations, and you should be able to write one sentence for each. The target population is the group the results are meant to describe, the source population is the listable group from which participants are actually drawn, and the study sample is the people who are selected, take part, and provide usable data; the sampling frame is the list that links the source population to the sample, and anyone missing from it cannot be selected. In the module's example of food insecurity among post-secondary students in British Columbia, the target is all post-secondary students in the province, the source is the students enrolled at institutions that share their enrolment lists, the frame is those lists, and the frame misses students at institutions that decline and students who enrol after the lists are drawn. Internal validity asks whether the results are correct for the source population, and external validity asks whether they generalise to the target population, so a gap in the frame threatens external validity, while non-response among people who were on the frame affects how well the study sample represents the source population. Because “study population” means the source population in some texts and the study sample in others, say which layer you mean.
  • 6.1 A probability sample gives every unit in the sampling frame a known, non-zero chance of selection through a formal random process (not a haphazard one), and that is what allows valid inference to the source population. The six designs differ in how units are chosen: a simple random sample draws individuals from one complete list; a systematic sample takes every jth unit after a random start (watch for periodic patterns in the list); a stratified sample divides the list into groups and samples from every group, which improves precision and guarantees each group is represented; a cluster sample chooses some groups at random and studies everyone in them, which is cheaper but less precise for the same number of people; a multistage sample chooses some groups and then samples individuals within each; and targeted sampling concentrates on the strata where disease is most likely, which is efficient for rare outcomes but needs prior knowledge of the risk factors. To name a described design, ask what is selected at each stage and how: if schools are chosen at random and children are then sampled within each chosen school, the design is a two-stage (multistage) cluster sample, while if everyone in the chosen schools is surveyed, it is a cluster sample. Hold on to the key contrast: a stratified design samples from every group, while a cluster design samples only some of the groups, so do not call a design stratified just because the population was divided into groups.
  • 6.2 Judgement, convenience, purposive, and chain-referral (snowball) samples are drawn without a formal random process from a sampling frame, so no one can say what share of the population each participant represents. That is why non-probability samples are generally unsuitable for estimating prevalence, although they are commonly used in analytic studies that compare exposure groups, and in hidden populations with no frame, where hybrids such as respondent-driven and time-location sampling partly recover probability-style inference. The direction of the bias can usually be predicted even when its size cannot: ask how the people easiest to reach differ from the target population on the outcome itself. A convenience sample in a walk-in clinic waiting room over-represents people who are ill or who readily seek care, so it would overestimate how common influenza-like illness is in the town, whereas a sample recruited at a fitness centre over-represents active people and would underestimate how common physical inactivity is. A complete answer names the design, explains why the selection probabilities are unknown, states the direction of the bias with its reason, and recognises that a larger sample would not remove the bias.
Knowledge Check: this section

1. A rural municipality of 1,800 households wants to know what proportion of its households have a working smoke alarm. Plan A sends inspectors to every household; Plan B sends them to a simple random sample of 300 households drawn from the municipal address list. Which statement is correct?

A census measures everyone, so its only error is in measurement; a sample adds sampling error, the chance difference between those selected and the whole population, yet a good sample can cost far less.
Counting every household removes sampling error but not measurement error. An inspector can still misjudge an alarm or record a household wrongly.
Random selection controls sampling error, not measurement error. Inspectors in Plan B can make the same measurement mistakes as in Plan A.
A well-planned probability sample can describe a whole population, with a stated margin of error. A census is not the only way to estimate a population proportion.
Correct answer: the census (Plan A) has measurement error only, and the sample (Plan B) has both measurement error and sampling error. Plan A is a census because every member of the population is measured, and Plan B is a sample because only a subset is measured. In a census the only error is in the measurement itself (an inspector who misjudges an alarm, a household recorded wrongly), while a sample adds sampling error, the chance difference between the 300 households selected and all 1,800. The second option is the tempting one, but a census still carries measurement error; random selection controls sampling error, not measurement error, and a well-planned sample can describe a population with a stated margin of error. (Section 3, 5.1 Census and Sample, including the callout on sources of error; Objective 5.1)

2. A health authority wants to estimate the proportion of adults aged 65 and over in its region who received an influenza vaccine this season. It draws a random sample of residents aged 65 and over from the provincial health-insurance client registry, and 640 of those sampled complete a telephone interview. Which group would this sampling frame miss, and which kind of validity does that gap threaten?

People who are selected but refuse are on the frame. Non-response affects how well the respondents represent the source population, which is a question of internal validity.
A random sample can represent only the people on its frame. Older residents without provincial coverage are not on the registry and cannot be selected.
Residents without provincial coverage are missing from the registry, so they lie outside the source population; the estimate may be right for registered residents but not generalise to all older adults.
The gap is real, but internal validity concerns the source population, here registered residents. People missing from the frame limit generalisation to the target population, which is external validity.
Correct answer: residents without provincial coverage, a threat to external validity. The information needed is the frame (the registry), the source population it defines (registered residents aged 65 and over), and the target population (all residents aged 65 and over in the region). People missing from the frame cannot be sampled, however good the interview is, so they fall outside the source population; internal validity asks whether the estimate is correct for the source population, and external validity asks whether it generalises to the target population, including the people the frame misses. The first option is tempting, but people who are selected and refuse are on the frame: non-response is a separate problem, which affects how well the study sample (the 640 respondents) represents the source population and so is a question of internal validity. (Section 3, 5.2.1 The Hierarchy of Populations and 5.2.2 The Sampling Frame, including the Canadian frames; Objective 5.2)

3. Two surveys are planned in the same region. Design A lists the region's 40 family-medicine clinics, selects 8 clinics at random, and then selects 25 patients at random from each selected clinic's patient roster. Design B divides the region's adults into urban and rural residents and draws a simple random sample of 500 adults from each group. Which names are correct?

This swaps the two designs. A stratified design samples from every group, as B does; a cluster design selects only some groups, as A does with 8 of 40 clinics.
A selects some clinics at random and then samples patients within each, a two-stage cluster sample; B samples from every group (urban and rural), a stratified random sample.
A one-stage cluster sample surveys everyone in the selected clusters. Design A draws 25 patients within each selected clinic, which adds a second stage.
Random selection of patients does not make A a simple random sample, because clinics are chosen first. B samples every group, which is stratification, not clustering.
Correct answer: A is a two-stage (multistage) cluster sample and B is a stratified random sample. Ask what is selected at each stage and how. In A, groups (clinics) are chosen at random first, and then a random sample of individuals is drawn within each chosen group, which makes it a two-stage cluster sample with clinics as the primary sampling units; in B, every group (urban and rural) is sampled, which makes it stratified. The third option is the tempting one, but a one-stage cluster sample surveys everyone in the selected clusters, whereas Design A samples 25 patients within each clinic; the first option swaps the two designs, forgetting that a stratified design samples from every group while a cluster design samples only some of the groups. (Section 3, 6.1.3 Naming the Design of a Described Study, Table 2.3 and the paragraph after it; Objective 6.1)

4. To estimate how common mobility limitations are among adults aged 65 and over in a city, a team interviews older adults walking in a large shopping mall on weekday mornings, approaching everyone who passes the survey table. Which statement about this sample is correct?

Approaching everyone who passes is not a probability sample, because only people who happen to be in the mall can be selected and their selection probability is unknown.
The design label is right but the direction is not. People walking through a mall are less likely to have a mobility limitation, so the estimate would be too low, not too high.
A larger convenience sample gives a more precise estimate of the wrong quantity. Size reduces random error but does not remove the bias from who is easiest to reach.
With no frame and no random process, selection probabilities are unknown; older adults able to walk in a mall are less likely to have a mobility limitation, so the sample underestimates prevalence.
Correct answer: a convenience sample that would underestimate the prevalence. The design is non-probability because there is no sampling frame and no formal random process, so no one can say what share of the city's older adults each participant represents. To predict the direction of the bias, ask how the people who are easiest to reach differ from the target population on the outcome itself: older adults who can walk through a mall are less likely to have a mobility limitation, so the sample over-represents people without one and the estimate is too low. Approaching everyone who passes does not make it a probability sample, because only people who happen to be in the mall can be selected, and a larger sample gives a more precise estimate of the wrong quantity rather than removing the bias. (Section 3, 6.2.2 Why Non-Probability Samples Cannot Estimate Prevalence, the callout and the paragraph after it; Objective 6.2)

5. A health authority wants to estimate the proportion of households in its region that have a 72-hour emergency kit. It obtains the residential address lists held by the region's municipalities, divides the addresses into urban and rural areas, and draws a simple random sample of 400 addresses from each area. Which design is this?

Dividing a population into groups does not make a cluster sample. A cluster design samples only some groups; here every group (urban and rural) is sampled, which is stratification.
The addresses are drawn at random, but separately within urban and rural groups, which is stratified sampling rather than one simple random sample of the whole region.
The frame is divided into urban and rural groups and a random sample is drawn from every group, which is stratified sampling and guarantees enough rural households for a separate estimate.
A multistage design selects groups at one stage and units within them at the next. Here addresses are drawn directly within each area, so there is a single stage of selection within strata.
Correct answer: a stratified random sample. The information needed is what is selected at each stage and how: the frame is divided into urban and rural groups, and a simple random sample is drawn from every group, which is stratified sampling. Sampling every group guarantees that each is represented and allows a separate estimate for each. A cluster sample would instead select only some groups at random (for example, some neighbourhoods) and survey households within them. The first option is tempting, but dividing a population into groups does not make a design a cluster sample; what matters is whether every group or only some groups are sampled. (Section 3, 6.1.2 The Six Probability Designs and 6.1.3 Naming the Design of a Described Study; Objective 6.1)

6. For the same survey, a colleague suggests handing out the questionnaire at the region's emergency-preparedness fair, where about 600 visitors are expected, because it would be quicker. Which statement is correct?

A larger sample only makes a biased estimate more precise. Visitors to a preparedness fair differ from households in general, and that bias remains whatever the size.
With no frame and no random process, selection probabilities are unknown; fair visitors are already interested in preparing, so they likely have kits more often, and size cannot fix that bias.
This reverses the likely bias. People who attend a preparedness fair are already interested in preparing, so they are more likely, not less, to have a kit.
Being free to attend is not a known, non-zero chance of selection. Only people who choose to come can be sampled, so this is a convenience sample.
Correct answer: a convenience sample that would probably overestimate the proportion. The information needed is how the sample is chosen and how the people easiest to reach differ on the outcome. There is no sampling frame and no formal random process, so each household's chance of selection is unknown and the result cannot be generalised to the region. People who come to an emergency-preparedness fair are already interested in preparing, so they are more likely than households in general to have a kit, and the estimate would be biased upward. The first option is tempting, but a larger sample only makes a biased estimate more precise, and the last option confuses being free to attend with having a known, non-zero chance of selection. (Section 3, 6.2.2 Why Non-Probability Samples Cannot Estimate Prevalence, the callout and the paragraph after it; Objective 6.2)

✦ Pass the knowledge check with 100% to continue

Section 4 of 5

Sampling Distributions and Survey Analysis

⏱ Estimated reading time: 40 minutes

Section 4 of 5

Sampling Distributions and Survey Analysis

Random variables, probability distributions, the central limit theorem, and the analysis of complex survey data.

Probability foundations

Random variables, expected value, variance

Random variable

A numerical outcome of a random process: e.g., the number of new TB cases in a health region next week.

Expected value (μ)

The long-run average across many repetitions. The quantity we are usually trying to estimate.

Variance (σ²)

How spread out the values are around the mean. Spread controls the precision of sample-based estimates.

Standard error of the mean
\[ \color{#0B7B6B}{SE} = \frac{\color{#C2410C}{\sigma}}{\sqrt{\color{#6D28D9}{n}}} \]
SE standard error of the meanσ population standard deviationn sample size
Key distributions

Probability distributions in public health

Discrete

Bernoulli(p): single yes/noBinomial(n, p): count of successesPoisson(λ): rare event counts

Continuous

Normal(μ, σ²): symmetric measurementsExponential(λ): skewed waiting timesUniform(a, b): equal probability over a range
The engine of inference

The Central Limit Theorem

Sampling distribution of the mean
\[ \color{#0B7B6B}{\bar{X}} \sim N\!\left(\color{#C2410C}{\mu},\; \frac{\color{#6D28D9}{\sigma^2}}{\color{#1D4ED8}{n}}\right) \text{ for large } n \]
X̄ sample meanμ population meanσ² population variancen sample size

No matter how skewed or irregular the underlying population, the distribution of sample means approaches Normal as n grows. This is what permits confidence intervals and hypothesis tests even with non-Normal raw data.

The interactive simulator below this walkthrough lets you watch the theorem in action across six different population shapes.

Three adjustments

Stratification, weights, clustering

Stratification

Combine stratum-specific estimates. Can reduce overall SE when the stratifying variable predicts the outcome.

Sampling weights

Weight = 1 / p(selection). Corrects for unequal selection probabilities. May change both point estimate and SE.

Clustering

Within-cluster similarity inflates variance. Adjust SE at the PSU level. Ignoring it, together with the unequal weights, produces SE values that are too small, typically by about 25–35% for CCHS health-region estimates.

Horvitz-Thompson weight
\[ \color{#0B7B6B}{w_i} = \frac{1}{\color{#C2410C}{\pi_i}} \]
wi survey weight for unit iπi selection probability of unit i
Quantifying the cost of complexity

The design effect (deff)

Design effect (Kish, 1965)
\[ \color{#0B7B6B}{\text{deff}} = \frac{\color{#C2410C}{\text{Var}_{\text{complex}}}}{\color{#6D28D9}{\text{Var}_{\text{SRS}}}} \]
deff design effectVarcomplex variance under the actual designVarSRS variance under simple random sampling

deff = 1

Complex design is as precise as SRS.

deff > 1

Less precise than SRS. Brazil diarrhea study: deff = 4.43. The variance was 4.43 times the SRS equivalent.

Two corrections

Finite population correction and bootstrap weights

Finite population correction (FPC)
\[ \color{#0B7B6B}{\text{FPC}} = \frac{\color{#C2410C}{N} - \color{#6D28D9}{n}}{\color{#C2410C}{N} - 1} \]
FPC finite population correctionN population sizen sample size

Apply when sampling fraction > 10%, in simple or stratified designs only. Not applicable in multistage sampling.

CCHS / CHMS practice: Statistics Canada releases 500 bootstrap replicate weights (Rao & Wu, 1988). Run your analysis 500 times, using survey-analysis software that supports replicate weights. Analysing the CCHS as if it were a simple random sample typically makes standard errors about 25–35% too small for health-region estimates, and more for some estimates, because the median design effect of these estimates is about 1.8 to 2.4, driven mostly by unequal selection probabilities and non-response adjustments (Chatrchi et al., 2015).

Carry forward

What to take into the next section

  • The standard error of a mean, σ/√n, falls with the square root of the sample size.
  • By the central limit theorem, sample means are approximately Normal in large samples, whatever the population's shape.
  • Weights correct for unequal selection probabilities; clustering inflates variance by the design effect.
  • An analysis that ignores the design reports estimates that look more precise than they are.

Overview

The designs in Section 3 decide who can be selected. This section explains why a small random sample can describe a whole population at all, and how a probability sample makes it possible to attach a measure of uncertainty to every estimate. It introduces random variables, expected values, and variances, surveys the probability distributions that describe most public-health data, and states the central limit theorem, which describes how sample means behave. It then shows how the stratification, unequal selection probabilities, and clustering of a complex survey enter the analysis, and how the design effect and the finite population correction summarize their effect on precision.

Learning Objectives

  • 7.1 Define a random variable, its expected value, and its variance, and explain why spread controls the precision of an estimate.
  • 7.2 Recognize common probability distributions (Bernoulli, Binomial, Poisson, Uniform, Normal, Exponential, log-normal) and the public-health phenomena they describe.
  • 7.3 Explain the sampling distribution of the mean, calculate a standard error, and state the central limit theorem in plain language.
  • 7.4 Explain how stratification, sampling weights, and clustering affect the analysis of survey data, and interpret a design effect and the finite population correction.

7.1 Random Variables, Expected Values and Variance

Every quantity we estimate from a sample, whether a prevalence, a mean, or a risk ratio, is the value of a random variable. If we drew a different sample tomorrow, the value would be slightly different. Probability theory is the formal language we use to describe how those values vary, and it is what lets us turn a single sample into a defensible statement about a population. Three concepts do most of the work in introductory biostatistics.

A random variable is a numerical outcome of a random process, for example the number of new TB cases reported in a health region next week, or the systolic blood pressure of the next adult who walks into a clinic. The expected value (also called the mean, written μ or E[X]) is the long-run average of a random variable across many repetitions of the random process. It is the value we are usually trying to estimate. The variance (σ2) and its square root, the standard deviation (σ), measure how spread out the random variable is around its mean. Spread, not the mean alone, is what controls how precisely we can estimate things from a sample: with the same sample size, the more individuals differ from one another, the more a sample mean varies from one sample to the next. Subsection 7.3 turns this into a formula, the standard error.

A fourth idea worth naming is independence. Two observations are independent when knowing the value of one tells you nothing about the value of the other. For example, the blood pressures of two strangers chosen at random from across a province are close to independent, whereas the blood pressures of two adults in the same household are not, because they share a diet, a neighbourhood, and often their genes. Independence is the assumption that lets a small random sample stand in for a much larger population, because it means that each new observation brings genuinely new information about the population. When observations are not independent, some of that information is repeated, and the sample behaves as if it were smaller than it is. This is the assumption most often violated in real public-health data (clustered households, repeated measures on the same person, contagion in infectious disease). Subsection 7.4 returns to the clustered case.

These ideas matter whenever a confidence interval, a hypothesis test, or a margin of error is reported. Each of those calculations is arithmetic on a probability distribution, usually a Normal distribution, that describes what would happen if you repeated the study many times. If you don’t know what distribution your statistic comes from, you can’t honestly attach uncertainty to it. Subsection 7.2 surveys the distributions that describe most public-health data.

7.2 Probability Distributions in Public Health

A probability distribution describes the values a random variable can take and how likely each value is. Every distribution is summarized by two things: a shape (symmetric? right-skewed? bimodal?) and a small set of parameters that control its location and spread. When you see "BMI ~ Normal(27, 42)", read it as: BMI is approximately Normal, centred at 27 kg/m2, with a standard deviation of 4, so about 95% of the population falls within 2 SDs of the mean, roughly 19 to 35. Distributions split first into discrete (countable outcomes such as 0, 1, 2 cases) and continuous (any value on a range, such as height, blood pressure, or time). The handful below are the ones you will encounter again and again in public health, and you choose among them by asking what process generated the values.

7.2.1 Discrete Distributions

Discrete distributions describe counts and yes/no outcomes. Table 2.4 sets out three of them, the Bernoulli, Binomial, and Poisson distributions, with what each models and a public-health example.

Table 2.4. Three discrete probability distributions, with what each models and a public-health example.

DistributionWhat it modelsPublic-health example
Bernoulli(p) A single yes/no trial with success probability p. Mean = p, variance = p(1−p). Whether one randomly chosen adult currently smokes.
Binomial(n, p) Number of "successes" in n independent Bernoulli trials. Mean = np, variance = np(1−p). Number of smokers in a CCHS sample of 1,000 adults.
Poisson(λ) Number of rare events in a fixed interval of time, area, or person-time. Mean = variance = λ. New cases of measles per week in a public-health unit; ER visits per hour.

The three are easiest to keep apart by the process behind them. A single yes/no outcome for one person is Bernoulli. The number of yes outcomes in a fixed number of independent trials, such as the number of smokers among 1,000 sampled adults, is Binomial. A count of rare, independent events in a fixed interval with no fixed upper limit, such as measles cases per week, is Poisson. The Binomial and the Poisson are the pair most often confused, since both describe counts; the Binomial has a fixed number of trials, while the Poisson does not.

7.2.2 Continuous Distributions

Continuous distributions describe measurements that can take any value on a range. Table 2.5 sets out the Uniform, Normal, Exponential, and log-normal distributions in the same format.

Table 2.5. Four continuous probability distributions, with what each models and a public-health example.

DistributionWhat it modelsPublic-health example
Uniform(a, b) Every value between a and b is equally likely. The "default ignorance" distribution. Random-digit dialling within an area code; random selection from a list.
Normal(μ, σ2) The classic bell curve: symmetric, with most mass within 2σ of the mean. The default for many continuous biological measurements and, importantly, for sample means (see CLT below). Adult height; systolic blood pressure; standardized test scores.
Exponential(λ) Time between independent events occurring at a constant rate λ. Right-skewed, memoryless. Mean = 1/λ. Time between successive ED arrivals; survival time under a constant hazard.
Log-normal / Right-skewed Variables that are positive and span several orders of magnitude. The log of the variable is approximately Normal. Household income; hospital length of stay; viral loads.

Here too the process points to the distribution. A symmetric biological measurement such as height or blood pressure is approximately Normal; the waiting time between events that occur at a constant rate, such as the time between emergency-department arrivals, is Exponential; a strictly positive, right-skewed quantity such as income, length of stay, or viral load is log-normal and is analysed on the log scale; and the Uniform distribution makes every value in a range equally likely, as in random-digit dialling.

7.2.3 Seeing and Exploring the Shapes

Figure 2.4 plots four of these distributions, two discrete and two continuous, so that their shapes can be compared directly.

Binomial(20, 0.3) Discrete · symmetric-ish Poisson(3) Discrete · counts of rare events Normal(μ, σ) Continuous · symmetric bell Exponential(λ) Continuous · right-skewed waits

Figure 2.4. Four common distributions you will meet in public-health data. Discrete distributions assign probability to whole-number outcomes (left two); continuous distributions describe a smooth curve over a real-valued measurement (right two).

Interactive 2.2 lets you build the same intuition by experiment. Use it to play with each distribution's parameters and watch the shape change in real time; the Distribution Explorer view also draws samples, so you can watch an empirical histogram converge on the theoretical curve as the sample grows. The Side-by-Side Comparison view plots six distributions on the same axes to help you remember which shape goes with which name. Its Central Limit Theorem view belongs with 7.3, and you can return to it after reading that subsection. Aim for at least 10 to 15 minutes of play; this is the kind of intuition you will draw on for every confidence interval and power calculation, and the main thing to take away is that each distribution describes a specific public-health phenomenon rather than serving as a textbook abstraction.

How to use this

Choose any of the six common distributions on the left. Adjust its parameters to see how the shape, mean, and spread respond. Then click "Draw a sample" to take a random draw and watch the empirical histogram converge on the theoretical curve as your sample grows.

Distribution

Number of successes in n independent yes/no trials. The natural model for "how many out of N have the condition?"

Parameters

Sampling

Tip: Bigger samples produce histograms that look more like the theoretical curve. The "law of large numbers" in action.
Theoretical distribution · Binomial(n = 20, p = 0.30)
The probability mass (discrete) or density (continuous) of the chosen distribution. Red dashed line marks the mean.
Theoretical mean (μ)
6.000
Theoretical SD (σ)
2.049
Sample mean (x̄)
–
Sample SD (s)
–

About Binomial: n = 20, p = 0.30

Binomial(n, p) counts the number of successes in n independent Bernoulli trials with the same success probability p.

Mean = np = 6.00, SD = √np(1−p) = 2.05.

  • For large n and moderate p, the binomial looks Normal; this is one of the oldest examples of the CLT.
  • Used for sample-size formulas around proportions and prevalence estimates.
  • Assumes independence and constant p; both can fail in clustered data (households, schools).

Central Limit Theorem demonstration

Pick a population shape (the heavily skewed ones show the effect most clearly). Set a sample size n, then click Draw 1,000 sample means. Each sample of size n is summarised by its mean, and we plot those means. Watch the histogram of means converge on a Normal curve as n grows, even when the underlying population is wildly non-Normal.

Population shape

The CLT says it does not matter which one of these you pick; sample means converge on a Normal curve.

Sample size

Each "trial" draws n values from the population and computes their mean.
Compare: Try n = 1 (the means just are the population). Then bump to n = 5, 30, 100. Notice the histogram becomes Normal-shaped and narrower.
Population distribution
This is the underlying "true" distribution we are sampling from. It can be anything: symmetric, skewed, or bimodal.
Population mean (μ)
–
Population SD (σ)
–
Mean of sample means
–
SD of sample means (SE)
–
Predicted SE = σ/√n
–
Trials drawn
0
Ready. Draw sample means to see the CLT in action.

Side-by-side: shapes at a glance

Six distributions, plotted on the same axes. Use this view to remember which shape goes with which name, and to compare how parameters change appearance. Hover for tooltips.

Display options

Different distributions can describe similar-looking data, so choosing the right one depends on the underlying mechanism as much as the shape.
Comparison of common distributions (rescaled to relative density)
Each curve is rescaled so peak density is 1, to make shape comparison easier. Shape, not absolute height, is what matters here.

How to choose a distribution in practice

  • Binary outcome (yes/no for one person)? Bernoulli. Count of "yes"s in a fixed sample? Binomial.
  • Counting rare events in time/space (cases per week, ER visits per hour)? Poisson.
  • Continuous, symmetric biological measurement (BP, height, lab values)? Normal, or check first whether the variable is approximately Normal.
  • Time until next event under a constant hazard? Exponential. Survival under varying hazard? Weibull or Gamma (advanced).
  • Strictly positive, right-skewed, multiplicative process (income, length of stay, viral load)? Log-normal; analyze on the log scale.
  • No prior information about likely values? Uniform on a sensible range.

Interactive 2.2. Distribution explorer for six common distributions, with views of the Central Limit Theorem and a side-by-side comparison of shapes.

Each distribution above describes the behaviour of a single random variable. When we draw a sample, however, the quantity we typically care about is a summary of that sample, such as a mean, proportion, or rate, and subsection 7.3 adds the layer of theory needed to make inferences from such summaries.

7.3 The Sampling Distribution of the Mean and the Central Limit Theorem

So far we have talked about distributions of individuals in a population. Now consider the distribution of a statistic, for example the mean of a random sample of n people. Because each sample of size n would give a slightly different mean, the mean itself has a distribution. We call it the sampling distribution of the mean. By this we mean the pattern you would see if you could repeat the same study many times, each time drawing a fresh random sample of the same size and recording its mean. Imagine, for example, drawing 1,000 different samples of 50 adults from the same city and calculating the average systolic blood pressure in each. Most of the 1,000 averages would sit close to the true city average, a few would land noticeably above or below it, and a histogram of them would show how much a single study's answer can be expected to wander by chance. In practice we draw only one sample, but the theory of the sampling distribution tells us how far that one answer is likely to be from the truth. Two facts about that sampling distribution drive almost all of frequentist inference.

The two pillars: standard error and the central limit theorem

1. Standard error. If individual observations have standard deviation σ, then sample means of size n have standard deviation σ/√n. This quantity, the SD of a statistic, is called the standard error (SE). Quadrupling your sample size halves the standard error. Keep the two ideas apart: the standard deviation tells you how much individual people differ from one another, while the standard error tells you how much a summary such as the mean would bounce around if you repeated the whole study.

2. The Central Limit Theorem (CLT). For sufficiently large n, the sampling distribution of the mean is approximately Normal, no matter what the underlying population looks like, even if the population is heavily skewed, bimodal, or discrete. "Sufficiently large" is often around n = 30 for moderately skewed distributions, and much smaller for symmetric ones.

The first pillar makes the point of 7.1 exact. Two simple random samples of 100 people whose standard deviations are 20 and 10 give standard errors of 20 ÷ √100 = 2 and 10 ÷ √100 = 1, so the first estimate is less precise even though the samples are the same size. The second pillar is the hidden engine behind the bell curve that shows up everywhere in statistics. It is the reason a 95% confidence interval can be written as estimate ± 1.96 · SE: the 1.96 comes from the Normal distribution that the CLT promises us applies to the estimate, even when we have no idea what shape the underlying population has. For example, a sample of 100 adults whose blood pressures have a standard deviation of 15 mmHg gives SE = 15 ÷ √100 = 1.5 mmHg, and a 95% confidence interval of the mean ± 1.96 × 1.5, or about ± 2.9 mmHg. The theorem traces from de Moivre (1733) through Laplace's 1812 binomial approximation to Lyapunov's 1901 general proof (Central limit theorem, Wikipedia).

Figure 2.5 illustrates the theorem with samples of increasing size drawn from a skewed population, and the Central Limit Theorem view of Interactive 2.2 in 7.2.3 lets you run the same experiment yourself, comparing the observed spread of the sample means with the predicted SE = σ/√n.

Population (skewed) e.g., household income draw samples of size n Means at n=2 Still skewed Means at n=10 Approaching Normal Means at n=50 Tightly Normal

Figure 2.5. The Central Limit Theorem in action. The population can be wildly skewed, but as n grows the sampling distribution of the mean becomes increasingly Normal and increasingly narrow (its SD = σ/√n).

The figure shows the means becoming Normal; it does not show the data becoming Normal, and confusing the two is the most common misreading of the theorem.

The CLT is about means, not individuals

A common student misconception is that “a large enough sample makes the data Normal.” It does not. Income data with 1,000 observations is still right-skewed. What becomes Normal is the distribution of the sample mean across hypothetical repeated samples; that is what we use to construct confidence intervals around the mean. Inference for medians, proportions, ratios, or extreme values relies on the CLT or its analogues in different ways and may need larger n or different methods (bootstrapping, exact methods).

The results above assume a simple random sample, in which every observation is independent and has the same chance of selection. Real surveys such as the CCHS combine stratification, unequal selection probabilities, and clustering, and subsection 7.4 shows how each of these features changes the analysis.

7.4 Analysing Complex Survey Data

7.4.1 Stratification, Sampling Weights and Clustering

When data come from a complex sampling design (involving stratification, weighting, or clustering), the analysis must account for these features. Ignoring them can lead to incorrect point estimates and underestimated standard errors.

Each feature has a distinct effect. Stratification mainly affects precision and can reduce the standard error when the stratifying variable is related to the outcome. Sampling weights, equal to 1 divided by each person's overall probability of selection, correct for unequal chances and can change the point estimate as well as the standard error. Clustering makes observations within a cluster alike, the failure of independence flagged in 7.1, so ignoring it gives standard errors that are too small. Open each tab below for how the analysis handles each feature, including Equation 2.1 for the probability of selection in a two-stage household survey.

Accounting for Stratification

If the population was divided into strata before sampling, this must be reflected in the analysis. Stratification provides stratum-specific estimates and can reduce the standard error of the overall estimate if the stratifying variable is related to the outcome. The reason is that the analysis then only has to account for variation within each stratum, because the share of the sample in each stratum was fixed by design and did not depend on chance. For example, if smoking is much more common in one health region than another, a stratified sample guarantees the right number of people from each region, so the overall estimate cannot be pushed up or down by drawing too many people from the high-smoking region by luck. If the stratifying variable has nothing to do with the outcome, there is little such variation to remove, and the gain in precision is small.

However, stratification alone does not change the overall point estimate; it primarily affects precision. The total population size in each stratum must be known to compute appropriate sampling weights.

Sampling Weights

Not all individuals in a probability sample necessarily have the same probability of selection. The sampling weight for each individual is the inverse of their overall selection probability; this inverse-probability weighting underlies the Horvitz-Thompson estimator introduced by Horvitz & Thompson (1952), which produces unbiased totals and means from any probability sample with known inclusion probabilities.

The probability of selection depends on multiple stages. For example, in a household survey:

p(selection) = (n/N) × (m/M)   (Equation 2.1)

where n = households in sample, N = households in source population, m = individuals selected per household, and M = total people in that household.

Multiplying the two stages captures a simple idea: your overall chance of being sampled is the chance your household is chosen times the chance you are then picked within it.

The sampling weight = 1/p(selection). This weight reflects how many people in the source population each sampled individual "represents." Incorporating weights may change both the point estimate and the standard error.

Accounting for Clustering

In cluster and multistage sampling, individuals within groups are usually more alike than randomly chosen individuals. This means observations are not independent, and standard errors must be adjusted upward. For example, if a survey interviews several adults in each of a few hundred households, and one household cooks with a smoky indoor stove, several members of that household may report respiratory symptoms together. A standard formula would treat those reports as separate pieces of evidence, when much of the information is shared, and it would therefore overstate how precise the estimate is. Adjusting the standard error upward corrects this overconfidence.

The most common approach is to identify the primary sampling unit (PSU) and adjust all standard error calculations for clustering at that level. The technique called variance linearisation is widely used for this purpose and requires a large number of PSUs to be reliable.

Equation 2.1 is applied by multiplying the probabilities of every stage first and then inverting the product. If a survey selects 500 of 50,000 households and then one of the four adults in a particular household, that adult's probability of selection is (500/50,000) × (1/4) = 0.0025, and the weight is 1/0.0025 = 400, so the adult stands for 400 people in the source population. Using only the household stage, or reporting the probability instead of the weight, are the usual slips.

7.4.2 The Design Effect

The design effect (deff) summarizes the overall impact of the sampling plan on precision. It is the ratio of the variance from the complex sampling design to the variance that would have been obtained from a simple random sample of the same size. The concept and the term were coined by Leslie Kish in his classic textbook Survey Sampling (Kish, 1965) and remain the standard summary statistic for complex-design efficiency. By this we mean that the design effect answers a single practical question: compared with the simplest possible design, how much precision did this sampling plan cost or gain? A deff of 1 means the complex design was exactly as precise as a simple random sample. A deff > 1 means the complex design produces less precise (larger variance) estimates than a simple random sample would, which is the usual result of clustering, because people in the same cluster carry overlapping information. A deff below 1, which can happen with effective stratification, means the design did better than a simple random sample. For example, in the Brazil diarrhea study, the deff was 4.43, meaning the variance of the incidence estimate was 4.43 times larger than what a simple random sample of the same size would have produced. Because the design effect multiplies the variance, the standard error grows by its square root, so a deff of 4 doubles the standard error.

Worked Example 2.3 uses the same study to trace how the incidence estimate and its standard error change as each feature of the sampling plan is taken into account, with the results set out in Table 2.6; watch how the point estimate and the standard error respond to different features.

Worked Example 2.3: Impact of Survey Design on Estimates

Table 2.6. Incidence estimate and standard error under an assumed simple random sample and as stratification, weights, and clustering are taken into account.

Type of AnalysisIncidence EstimateSE
Simple random sample (assumed)0.14620.0061
+ Stratification0.14620.0059
+ Stratification + Weights0.17510.0091
+ Clustering0.14620.0088
All features combined0.17510.0128

Notice how incorporating all features of the sampling plan changes both the point estimate (from 14.62% to 17.51%) and dramatically increases the standard error (from 0.0061 to 0.0128). Ignoring the sampling design would give a misleadingly precise, and potentially incorrect, result.

Worked Example 2.3 showed that ignoring the design misstates both the estimate and its precision. The rows of Table 2.6 match the three features of 7.4.1: stratification alone trims the standard error slightly, weights move the point estimate, and clustering inflates the standard error. Statistics Canada builds the same corrections into its own surveys. It distributes the CCHS and CHMS with a set of 500 bootstrap replicate weights rather than releasing the underlying cluster identifiers (which would risk re-identification). The rescaling-bootstrap method that produces these weights was developed by Rao & Wu (1988). To get correct standard errors you re-run your analysis 500 times, once with each replicate weight, and combine the results. Most statistical packages include survey-analysis procedures that support replicate weights and combine the 500 results automatically. If you ignore the bootstrap weights and just analyse the CCHS as if it were a simple random sample, your standard errors will typically be about 25–35% too small (more for some estimates), and your confidence intervals and p-values become meaningless.

7.4.3 The Finite Population Correction

One feature of a design can improve precision rather than reduce it. When the proportion of the population sampled is relatively large (>10%), precision improves beyond what would be expected from an "infinite" population, and the finite population correction (FPC) adjusts the estimated variance downward by the factor in Equation 2.2:

FPC = (N − n) / (N − 1)   (Equation 2.2)

where N is the population size and n is the sample size. The intuition is straightforward: once you have already measured a large share of the population, little of it is left to be uncertain about, so the estimate is more precise than the standard (infinite-population) formula assumes. For example, sampling 500 of the 1,000 nurses in a health authority gives FPC = (1,000 − 500) / (1,000 − 1) = 500 / 999, or about 0.50, so the variance is roughly halved. Sampling 500 people from a city of a million, by contrast, gives a factor so close to 1 that the correction makes no practical difference, which is why it is ignored for small sampling fractions. The FPC is only applicable to descriptive studies using simple or stratified random sampling, and it should not be applied in multistage sampling even if the number of PSUs sampled exceeds 10% of the total PSUs. Section 5 uses the same correction, together with the design effect, when planning how large a sample needs to be.

This section has shown why probability sampling supports inference, from random variables and distributions through the Central Limit Theorem to the analysis of complex survey data. The key takeaways and knowledge check that follow review these points.

Key Takeaways

  • 7.1 Every estimate you calculate from a sample, whether a prevalence, a mean, or a risk ratio, is the value of a random variable: draw a different sample and you would get a slightly different number. The expected value (μ) is the long-run average of that variable, the quantity you are usually trying to estimate, and the variance (σ²) and its square root, the standard deviation (σ), describe how spread out the values are around it. Spread, not the size of the mean, controls precision: with the same sample size, the more individuals differ from one another, the more sample means vary from one sample to the next. For example, two simple random samples of 100 people with standard deviations of 20 and 10 give standard errors of 20 ÷ √100 = 2 and 10 ÷ √100 = 1, so the first estimate is less precise even though the samples are the same size. These results also assume independent observations, the assumption most often violated in real data by clustered households, repeated measures on the same person, and contagion.
  • 7.2 A small set of probability distributions describes most public-health data, and you choose among them by asking what process generated the values. A single yes/no outcome for one person is Bernoulli; the number of yes outcomes in a fixed number of independent trials, such as smokers among 1,000 sampled adults, is Binomial; a count of rare, independent events in a fixed interval with no fixed upper limit, such as measles cases per week, is Poisson; a symmetric continuous measurement such as height or blood pressure is approximately Normal; the waiting time between events that occur at a constant rate, such as the time between emergency-department arrivals, is Exponential; a strictly positive, right-skewed quantity such as income, length of stay, or viral load is log-normal and is analysed on the log scale; and the Uniform distribution makes every value in a range equally likely, as in random-digit dialling. The most common error is to swap the Binomial and the Poisson: both describe counts, but the Binomial has a fixed number of trials, while the Poisson counts events with no fixed upper limit. Keep the Bernoulli (one person's outcome) separate from the Binomial (the count across many people) as well.
  • 7.3 If you repeated a study many times, the sample means would form a distribution of their own, the sampling distribution of the mean. Its standard deviation is the standard error, SE = σ/√n, so quadrupling the sample size halves the standard error; the standard deviation describes how much individuals differ, while the standard error describes how much the mean would vary from one study to the next. The central limit theorem says that, for a sufficiently large sample (often about 30 for moderately skewed data), the sampling distribution of the mean is approximately Normal whatever the shape of the population, which is why a 95% confidence interval can be written as the estimate ± 1.96 × SE. For example, a sample of 100 adults whose blood pressures have a standard deviation of 15 mmHg gives SE = 15 ÷ √100 = 1.5 mmHg, and a 95% confidence interval of the mean ± 1.96 × 1.5, or about ± 2.9 mmHg. Watch for two traps: dividing by n instead of √n, and claiming that a large sample makes the data themselves Normal, when only the distribution of the mean becomes Normal and skewed data such as incomes stay skewed however many people are surveyed.
  • 7.4 Real surveys such as the CCHS combine stratification, unequal selection probabilities, and clustering, and the analysis must reflect all three or its answer will mislead. Stratification mainly affects precision and can reduce the standard error when the stratifying variable is related to the outcome; sampling weights, equal to 1 divided by each person's overall probability of selection, correct for unequal chances and can change the point estimate as well as the standard error; and clustering makes people within a cluster alike, so ignoring it gives standard errors that are too small. To calculate a weight, multiply the probabilities of every stage first, p(selection) = (n/N) × (m/M), and then invert the result; using only the household stage, or reporting the probability instead of the weight, are the usual slips. The design effect (deff) is the ratio of the design's variance to the variance of a simple random sample of the same size, so a deff above 1 means less precision, and because it multiplies the variance, the standard error grows by its square root (a deff of 4 doubles the standard error). The finite population correction, (N − n)/(N − 1), reduces the variance when more than about 10% of a population is sampled, but it applies only to descriptive estimates from simple or stratified random samples, not to multistage designs.
Knowledge Check: this section

1. Two health regions each survey a simple random sample of 100 adults to estimate mean daily sodium intake. In Region A the standard deviation of intake is 1,200 mg; in Region B it is 600 mg. Which statement is correct?

Sample size is only half of the standard error formula, σ ÷ √n. With the same n, the larger spread in Region A gives a larger standard error.
A larger spread makes each sample mean vary more from sample to sample, so it lowers precision. Region A's standard error is twice Region B's.
Precision depends on the spread and the sample size, not on the size of the mean. The standard error σ ÷ √n contains no term for the mean itself.
The standard error of a mean is σ ÷ √n: 1,200 ÷ 10 = 120 mg in Region A and 600 ÷ 10 = 60 mg in Region B, so the region with more spread has the less precise mean.
Correct answer: Region A's mean is less precise, with a standard error of 120 mg against 60 mg. The daily sodium intake of a randomly chosen adult is a random variable; its expected value, μ, is the population mean the surveys are trying to estimate, and its variance, σ², describes how spread out intakes are around that mean. The standard error of a sample mean is σ ÷ √n, so with the same n = 100 (√100 = 10), Region A's standard error is 1,200 ÷ 10 = 120 mg and Region B's is 600 ÷ 10 = 60 mg. The first option is the tempting one, but sample size is only half of the formula: when individuals differ more, each sample mean varies more from sample to sample, so the spread, not the mean, controls precision. (Section 4, 7.1 Random Variables, Expected Values and Variance, and the standard-error pillar in 7.3; Objective 7.1)

2. An analyst is choosing a probability distribution for four public-health variables: (1) the number of new Lyme disease cases reported each week in a health unit, where cases are rare and occur independently; (2) the number of current smokers among 500 randomly sampled adults; (3) the time between successive ambulance calls overnight; (4) the systolic blood pressure of adults. Which matching is correct?

This swaps Poisson and Binomial. Smokers among 500 adults is a count of yes outcomes in a fixed number of trials (Binomial); rare weekly cases with no fixed upper limit are Poisson.
Rare independent events per interval are Poisson, yes outcomes among a fixed number of trials are Binomial, waiting times between events are Exponential, and blood pressure is approximately Normal.
Waiting time between events is right-skewed and Exponential, while a symmetric continuous biological measurement such as blood pressure is approximately Normal. This option swaps them.
A Bernoulli variable describes one person's yes or no outcome. The count of smokers across 500 sampled adults is Binomial.
Correct answer: Poisson, Binomial, Exponential, Normal. Choose each distribution from the mechanism that generates the data. A count of rare, independent events in a fixed interval of time (cases per week) is Poisson; a count of "yes" outcomes in a fixed number of independent yes/no trials (smokers among 500 adults) is Binomial; a waiting time between events that occur at a roughly constant rate is Exponential, which is right-skewed; and a symmetric continuous biological measurement such as blood pressure is approximately Normal. The most common error is to swap Poisson and Binomial: both describe counts, but the Binomial has a fixed number of trials (500 people), while the Poisson counts events with no fixed upper limit; a Bernoulli variable describes one person's yes/no outcome, not the count across 500 people. (Section 4, 7.2 Probability Distributions in Public Health, Tables 2.4 and 2.5; Objective 7.2)

3. A simple random sample of 400 households finds a mean annual household income of $68,000, with a standard deviation of $40,000. Household incomes are strongly right-skewed. Which statement is correct?

The standard error is 40,000 ÷ √400 = $2,000, and by the central limit theorem the sample mean is approximately Normal for a large sample whatever the population shape, giving 68,000 ± 3,920.
The central limit theorem concerns the distribution of the sample mean across repeated samples. Individual incomes stay skewed however many households are surveyed.
The central limit theorem applies whatever the shape of the population, provided the sample is large. With n = 400, the mean is approximately Normal and an interval can be calculated.
The standard error divides by √n, not n: 40,000 ÷ 20 = $2,000, not $100. Dividing by n greatly overstates precision.
Correct answer: the mean is approximately Normal, with a standard error of $2,000 and a 95% interval of about $64,100 to $71,900. The information needed is the sample size (400), the standard deviation ($40,000), and the fact that the sample is large. The standard error is σ ÷ √n = 40,000 ÷ 20 = $2,000, and by the central limit theorem the sampling distribution of the mean is approximately Normal for a sample this large, whatever the shape of the population, so the interval is 68,000 ± 1.96 × 2,000 = 68,000 ± 3,920. The second option is the most common misreading of the theorem: it concerns the distribution of the sample mean across repeated samples, not the individual incomes, which stay skewed however many households are surveyed; dividing by n instead of √n gives the wrong standard error. (Section 4, 7.3 The Sampling Distribution of the Mean and the Central Limit Theorem, including its two callouts; Objective 7.3)

4. A household survey selects 200 of a town's 8,000 households at random and then selects one adult at random in each selected household. What is the sampling weight of an adult who lives in a selected household with 3 adults?

40 uses only the household stage. An adult in a three-adult household had a one-in-three chance of being chosen, so the selection probability is 1/40 × 1/3 = 1/120.
0.0083 is the overall probability of selection, 1/120. The sampling weight is its reciprocal, 120.
The selection probability multiplies the two stages, (200/8,000) × (1/3) = 1/120, and the weight is its reciprocal, so this adult stands for about 120 adults in the town.
13.3 divides 40 by 3. The probabilities of the two stages multiply, so the weight is 40 × 3 = 120.
Correct answer: 120. The weight is 1 divided by the overall probability of selection, and the probability multiplies the two stages: p(selection) = (n/N) × (m/M) = (200/8,000) × (1/3) = 1/40 × 1/3 = 1/120, or about 0.0083, so the weight is 120, meaning this adult stands for about 120 adults in the town. The most tempting error is 40, which uses only the household stage (1 ÷ 1/40) and forgets that an adult in a three-adult household had only a one-in-three chance of being the person chosen; 0.0083 is the probability, not the weight, and 13.3 divides 40 by 3 instead of multiplying. Ignoring these weights would over-represent adults from small households, which is why weights can change the point estimate as well as the standard error. (Section 4, 7.4.1 Stratification, Sampling Weights and Clustering, the Sampling Weights tab, Equation 2.1 and the paragraph after the tabs; Objective 7.4)

5. A health authority selects 30 Grade 9 classes at random and surveys every student in them, 900 students in all. Treating the students as a simple random sample, the standard error of their mean screen time is 0.10 hours. The design effect for this mean is 1.96. What is the standard error once the design is allowed for?

A design effect above 1 means the clustered design is less precise than a simple random sample. Students in the same class resemble one another, so the standard error grows.
The design effect is a ratio of variances, so it multiplies the variance and the standard error grows by its square root: 0.10 × √1.96 = 0.10 × 1.4 = 0.14 hours.
This applies a ratio of variances directly to a standard error. The standard error is multiplied by √1.96 = 1.4, not by 1.96.
This has clustering backwards. A design effect above 1 always means less precision, so the standard error increases rather than shrinking.
Correct answer: 0.14 hours. The information needed is what the design effect measures: it is the ratio of the variance under the actual design to the variance under a simple random sample of the same size, so it multiplies the variance, and the standard error grows by its square root: 0.10 × √1.96 = 0.10 × 1.4 = 0.14 hours. Students in the same class resemble one another, so the clustered sample is less precise than a simple random sample of 900. The third option is the tempting error, because it applies a ratio of variances directly to a standard error, and the last option has the effect of clustering backwards: a design effect above 1 always means less precision. (Section 4, 7.4.2 The Design Effect; Objective 7.4)

✦ Pass the knowledge check with 100% to continue

Section 5 of 5

Planning Sample Size

⏱ Estimated reading time: 40 minutes

Section 5 of 5

Planning Sample Size

Type I and Type II error, power, the sample-size formulae, and their adjustments.

Two ways to be wrong

Type I and Type II error

Type I error (α)

Reject the null when it is true. A false positive. Conventional threshold: α = 0.05.

Type II error (β)

Fail to reject the null when it is false. A false negative. Usually set at β = 0.20.

Decision matrix
\[ \begin{array}{l|cc} & \text{Effect present} & \text{No effect} \\ \hline \text{Reject } H_0 & \text{Correct} & \color{#C2410C}{\alpha}\text{ error} \\ \text{Fail to reject} & \color{#6D28D9}{\beta}\text{ error} & \text{Correct} \end{array} \]
α error Type I: reject a true null (false positive)β error Type II: fail to reject a false null (false negative)
Power

Power = 1 − β

Statistical power
\[ \color{#0B7B6B}{\text{Power}} = 1 - \color{#6D28D9}{\beta} \]
Power probability of detecting a true effectβ Type II error rate

A study with 80% power (β = 0.20) will detect a true effect of the specified size eight times out of ten.

  • To increase power: increase sample size, reduce measurement error, or target a larger effect size.
  • A negative finding is only informative if power was adequate. Most underpowered studies are inconclusive, not reassuring.
Four formulae

Key sample-size formulae

Estimate a proportion
\[ \color{#0B7B6B}{n} = \frac{\color{#C2410C}{Z_{\alpha}^{2}}\, \color{#6D28D9}{p}\, \color{#1D4ED8}{q}}{\color{#BE185D}{L^{2}}} \]
n required sample sizeZα z-value for the confidence levelp expected proportionq 1 − pL margin of error
Estimate a mean
\[ \color{#0B7B6B}{n} = \frac{\color{#C2410C}{Z_{\alpha}^{2}}\, \color{#6D28D9}{\sigma^{2}}}{\color{#BE185D}{L^{2}}} \]
n required sample sizeZα z-value for the confidence levelσ² population varianceL margin of error
Compare two proportions (per group)
\[ \color{#0B7B6B}{n} = \frac{\left[\color{#C2410C}{Z_{\alpha}}\sqrt{2\bar{p}\bar{q}} - \color{#6D28D9}{Z_{\beta}}\sqrt{p_1 q_1 + p_2 q_2}\right]^{2}}{(\color{#1D4ED8}{p_1} - \color{#BE185D}{p_2})^{2}} \]
n sample size per groupZα z-value for significanceZβ z-value for powerp1 proportion in group 1p2 proportion in group 2
Clustering inflates n

Design-effect and FPC adjustments

Clustering adjustment
\[ \color{#0B7B6B}{n'} = \color{#C2410C}{n} \times \bigl[1 + \color{#6D28D9}{\rho}\,(\color{#1D4ED8}{m} - 1)\bigr] \]
n' cluster-adjusted sample sizen unadjusted sample sizeρ intracluster correlationm average cluster size
Finite population correction adjustment
\[ \color{#0B7B6B}{n'} = \frac{1}{\dfrac{1}{\color{#C2410C}{n}} + \dfrac{1}{\color{#6D28D9}{N}}} \]
n' corrected sample sizen uncorrected sample sizeN population size

Brazil example: unadjusted n = 685 per group → after clustering (ρ = 0.45, m = 6) → n = 2,227 per group. More than three times the unadjusted estimate.

Lesson summary

Bringing the lesson together

  • Surveillance supplies the signal and the data; outbreak investigation is the structured response.
  • The frame and the design decide who can be sampled and what the estimates can claim.
  • The central limit theorem and the design effect turn a sample into an estimate with honest uncertainty.
  • A sample-size calculation records its assumptions, and the result is padded for non-response.

Overview

The last section concerns the planning stage of a study. A study that is too small may miss a real effect, and a study that is too large wastes resources and burdens participants. This section defines the two types of statistical error and statistical power, explains how to choose between a calculation for precision and one for power and how the four statistical inputs drive the answer, and provides the formulae and calculators for estimating a proportion or a mean and for comparing two groups. It ends with the adjustments for clustering, for a small population, and for non-response.

Learning Objectives

  • 8.1 Explain Type I and Type II errors and statistical power, and explain why a negative result from an underpowered study is uninformative.
  • 8.2 Choose between a sample-size calculation for precision and one for power, and describe how precision, expected variation, confidence level, and power determine the required sample size.
  • 8.3 Calculate sample sizes for estimating a proportion or a mean and for comparing two proportions or two means.
  • 8.4 Adjust a sample size for a finite population, for clustering through the design effect, and for expected non-response.

8.1 Type I and Type II Errors and Statistical Power

In any study based on a sample, the variability of the outcome, measurement error, and sample-to-sample variability all affect results, so inferences based on sample data are subject to error. Within hypothesis testing in analytical studies, there are two key types of error, and Table 2.7 shows where each arises by crossing the conclusion of an analysis with whether the effect is truly present.

Table 2.7. The conclusion of an analysis crossed with the true state of the effect, showing the two correct conclusions and the Type I and Type II errors.

Conclusion of AnalysisEffect Truly PresentEffect Truly Absent
Effect present (reject null)CorrectType I (α) error
No effect (accept null)Type II (β) errorCorrect

A Type I error (probability α) is a false positive: concluding that an effect exists when it does not, by falsely rejecting the null hypothesis; α is conventionally set at 0.05. A Type II error (probability β) is a false negative: missing an effect that is really there; β is usually set at 0.20. Statistical power, 1 − β, is the probability of detecting a real effect of a stated size. Expand the three panels below for a fuller definition of each, the reasons a study can miss a real effect, and why negative findings are reported less often.

Type I (α) Error ▼

A Type I error occurs when you conclude that the outcomes in the groups are different (i.e., that an association exists), when in fact they are not. In other words, you falsely reject the null hypothesis. The probability of a Type I error is denoted α.

Statistical tests are aimed at disproving the null hypothesis (that there is no difference between groups). When P ≤ 0.05, we are "reasonably sure" that any detected effect is not due to chance, but there remains a 5% chance of making a Type I error. More precisely, setting α at 0.05 means that if there were truly no difference between the groups, a study of this kind would still produce a "significant" result about 5 times in every 100 repetitions, purely through the luck of who was sampled. For example, if 100 independent studies each tested a vitamin that had no real effect, about 5 of them would be expected to report that it worked.

Type II (β) Error ▼

A Type II error occurs when you conclude that there is no association between the exposure and outcome, when in fact there is. You fail to reject the null hypothesis when you should have. The probability of a Type II error is denoted β.

Reasons a study might fail to find a real effect include: the exposure truly had no effect, the study design was inappropriate, the sample size was too small (low power), or simply bad luck.

Statistical Power ▼

Power is the probability that you will find a statistically significant difference when a real difference of a defined magnitude exists. Mathematically, power = 1 − β.

For example, if a study has 80% power, it has an 80% chance of detecting a true effect of the specified size. To increase power, you need to increase the sample size. So-called negative findings (failure to find a difference) are less commonly reported in the literature, partly because many studies lack adequate power.

The panels show that the main way to increase power is to increase the sample size. This is why a negative result from an underpowered study is inconclusive rather than reassuring: a study with 40% power has β = 0.60 and would miss a real effect of that size six times in ten, so its failure to reject the null hypothesis says little about whether the effect exists. Failing to reject the null hypothesis never proves that it is true. When you name the error in a scenario, remember that a Type I error needs a positive finding and a Type II error a negative one.

The link between sample size and power runs through precision: a larger sample gives a narrower sampling distribution of the mean, the standard error of 7.3, so a real difference is less likely to be hidden by chance variation. Interactive 2.3 shows how the spread of sample means narrows as the sample size grows.

🎲 Interactive 2.3: Sample Size & the Law of Large Numbers

What you'll do: pick a population, set a sample size n, draw repeated samples, and watch the distribution of sample means concentrate around the true population mean as n grows. What to take away: the standard error shrinks as 1/√n; this is the formal reason why “more data” means better precision and higher statistical power. The intuition you build here drives every confidence interval and sample-size calculation in the rest of the course.

Population Distribution

The "true" distribution we're sampling from. Red line = true mean (μ).

0.506.50μ=3.50Fair die
Sampling Distribution of the Mean

Each bar is the count of sample means falling in that range. Yellow = most recent sample's mean.

Draw samples to see distribution of means0.506.50Sample mean (x̄)
True mean (μ)
3.50
Mean of sample means
–
Std. error (observed)
–
Theoretical SE = σ/√n
0.54
Samples drawn
0
Last sample mean
–
Try this: set n = 1, draw 100 samples, then bump n to 50 and draw 100 more (after Reset). The spread of the yellow histogram shrinks dramatically, the 1/√n precision gain in action.

Power, precision, and confidence are therefore design choices that are fixed before any data are collected. Subsection 8.2 shows how to decide which of them a study's sample size should be built on.

8.2 Precision or Power: Choosing the Calculation

Choosing the right sample size involves both statistical and non-statistical considerations. Non-statistical factors include available resources (time, money, personnel) and the nature of the sampling frame. Among the statistical considerations, the first decision is which kind of calculation the study needs, and it follows from the study's aim.

Precision or power: which calculation?

A precision calculation is used for a descriptive aim: estimating one quantity, such as a prevalence or a mean, in one population. It asks how many people are needed so that the confidence interval around the estimate is no wider than a chosen half-width L (for example, ±4 percentage points with 95% confidence). Its inputs are the expected proportion (or the variance of a measurement), the allowable error L, and the confidence multiplier Zα.

A power calculation is used for an analytic aim: comparing two or more groups. It asks how many people are needed so that the study has a stated probability (the power, 1 − β, usually 80%) of detecting a difference or association of a stated size if that difference is real. It adds two inputs to those above: the power multiplier Zβ and the size of the difference to be detected (for example, a fall in risk from 15% to 10%). A study with both a descriptive and an analytic aim calculates both and uses the larger number.

In an answer, say why the aim is descriptive or analytic before you name the calculation. The calculation then combines up to four statistical inputs, and each pushes the sample size in a predictable direction.

The four inputs are the precision required (the allowable error L, half the width of the confidence interval), the expected variation in the data (pq for a proportion, σ² for a measurement), the level of confidence (which sets Zα, 1.96 for 95%), and, for analytic studies only, the power (which sets Zβ, −0.84 for 80%). Expand each panel below for how each input is chosen in practice, including a quick way to estimate σ when only a plausible range of values is known.

Precision of the Estimate ▼

The more precise you need your estimate to be, the larger the sample you need. If you want to know diarrhea prevalence within ±5%, you need more subjects than if ±10% is acceptable. Precision is denoted L (the "allowable error" or half the desired confidence interval width).

Expected Variation in the Data ▼

The more people differ from one another on the outcome, the more a sample estimate will vary by chance, and so the more people are needed to pin it down. For proportions, variance = p × q (where q = 1 − p). This product is largest when p = 0.5, because a population split half and half is as mixed as a yes/no outcome can be, and it shrinks as p moves toward 0 or 1. For example, pq is 0.5 × 0.5 = 0.25 when half the population has the condition, but only 0.1 × 0.9 = 0.09 when one in ten does. You need a rough estimate of the proportion to calculate the required sample size, which can come from an earlier survey, a pilot study, or the published literature. For continuous variables like BMI, you need an estimate of the population variance (σ²). One approach: estimate the range that covers 95% of values, divide by 4 to get σ, then square it for σ². This works because, for a roughly Normal measurement, about 95% of values lie within 2 standard deviations either side of the mean, so the full range covers about 4 standard deviations.

Level of Confidence ▼

The confidence level (typically 95%) determines how sure you want to be that the confidence interval includes the true population value. This is linked to the Z-value: for 95% confidence, Zα = 1.96. Higher confidence requires a larger sample.

Power (for Analytic Studies) ▼

In analytical studies, you also need to specify the desired power (often 80%). Power determines the sample size needed to detect a specific effect size. For 80% power, Zβ = −0.84. Greater power requires a larger sample.

In summary, a narrower margin, more variation, a higher confidence level, more power, or a smaller difference to detect all increase the sample size. Subsection 8.3 turns these inputs into numbers.

8.3 Sample-Size Formulae and Calculators

The four statistical inputs combine into a handful of formulae, one for each planning objective. Interactive 2.4 contains the formulae, Equations 2.3 to 2.8, and a calculator for each: four planning objectives (estimating a proportion or a mean, and comparing two proportions or two means) and two adjustments, for a small population and for clustered sampling, which 8.4 explains. Its opening panel explains the two Z-values: Zα is the confidence multiplier from 8.2, and Zβ is the power multiplier, written with its negative sign so that subtracting it adds to the sample size. Choose the objective that matches the study question, read the formula and its key, then enter your own assumptions in the calculator. Colours link every symbol to its meaning: hover over any term in the key, in the formula, or in the calculator to trace it. Teal is always the answer, orange is always Zα, and violet is always Zβ.

What the Z-values mean

Z is a position on the standard normal curve (mean 0, standard deviation 1), measured in standard deviations from the centre. Two Z-values appear in the formulae, and both are read off the same curve.

Zα is the confidence multiplier. It is the number of standard errors on either side of an estimate that captures the chosen confidence level. For a two-sided 95% confidence interval, 95% of the curve lies between −1.96 and +1.96, so Zα = 1.96. A higher confidence level means a larger Z and a larger sample.

Zβ is the power multiplier, used only when comparing groups. It is the point that cuts off the lower β of the curve, where β is the Type II error rate (1 − power). For 80% power, β = 0.20 and Zβ = −0.84. Following the textbook convention, Zβ is written with its negative sign, which is why the formulae subtract it: subtracting a negative number adds to the numerator, so more power means a larger sample.

Zα: confidence level
−1.96+1.96095% of the curve
ConfidenceαZα
90%0.101.645
95%0.051.96
99%0.012.576
Zβ: statistical power
−0.840lower 20% (β)
PowerβZβ
80%0.20−0.84
90%0.10−1.28
95%0.05−1.645

Every formula gives a minimum. Always round n up to the next whole number, treat it as the number of people with usable data, and add a buffer for non-response and drop-out (typically 10 to 25%). Build the buffer by dividing n by the proportion you expect to respond, not by adding a percentage: with 80% expected to respond, 385 ÷ 0.80 = 482, whereas adding 20% gives only 462, which falls short.

Estimate a proportion

Objective. Report how common something is in one population (a prevalence, a proportion, a percentage) with a confidence interval no wider than you can tolerate. This is the formula for a descriptive survey with a yes/no outcome: What proportion of households have a rainwater cistern? What share of adults had diarrhea in the past month?

Before you start you need a rough guess of the proportion (from a pilot study or an earlier survey; if you have no idea, use 0.5, which gives the largest possible n), the amount of error you can accept on either side of the estimate, and a confidence level.

Equation 2.3. Sample size to estimate a proportion
\[ {\color{#0B7B6B}{n}} = \frac{{\color{#C2410C}{Z_{\alpha}^{2}}} \; {\color{#1D4ED8}{p}} \; {\color{#1D4ED8}{q}}}{{\color{#BE185D}{L^{2}}}} \]
sample size = (confidence multiplier)² × expected proportion × (1 − expected proportion) ÷ (allowable error)²
Every term, defined
  • nRequired sample size: the number of people you need to measure.
  • ZαConfidence multiplier from the standard normal curve.1.96 for 95% confidence (1.645 for 90%, 2.576 for 99%).
  • p, qp is the proportion you expect to find, as a decimal; q = 1 − p is its complement. Their product pq is the variance of a proportion, largest when p = 0.5.15% expected prevalence: p = 0.15, q = 0.85.
  • LAllowable error: half the width of the confidence interval you want, in the same decimal units as p.Plus or minus 5 percentage points: L = 0.05. Halving L quadruples n.
Calculator
As a decimal: 15% is 0.15. q is calculated for you.
As a decimal: ±5 percentage points is 0.05.
Required sample size–

Estimate a mean

Objective. Estimate the average of a continuous measurement in one population (mean BMI, mean systolic blood pressure, mean days of illness) to within a chosen margin. This is the formula for a descriptive study with a measured outcome.

Before you start you need an estimate of how spread out the measurement is in the population (its standard deviation σ; if all you know is a plausible range that covers about 95% of people, divide that range by 4 to get σ), the margin of error you can accept in the same units as the measurement, and a confidence level.

Equation 2.4. Sample size to estimate a mean
\[ {\color{#0B7B6B}{n}} = \frac{{\color{#C2410C}{Z_{\alpha}^{2}}} \; {\color{#1D4ED8}{\sigma^{2}}}}{{\color{#BE185D}{L^{2}}}} \]
sample size = (confidence multiplier)² × population variance ÷ (allowable error)²
Every term, defined
  • nRequired sample size: the number of people you need to measure.
  • ZαConfidence multiplier from the standard normal curve.1.96 for 95% confidence (1.645 for 90%, 2.576 for 99%).
  • σ²Population variance: the standard deviation of the measurement, squared. The more people vary, the more of them you need.Systolic blood pressure with SD 14 mmHg: σ² = 196.
  • LAllowable error: half the width of the confidence interval you want, in the units of the measurement.A mean within ±2 mmHg: L = 2.
Calculator
Enter σ; it is squared for you. Range ÷ 4 is a rough guide.
Same units as the measurement (mmHg, kg/m², days).
Required sample size–

Compare two proportions

Objective. Test whether the proportion with an outcome differs between two groups (exposed and unexposed, intervention and control), with enough people to detect a difference of a stated size if it is real. This is the formula for an analytic study with a yes/no outcome: a cohort study, a comparative cross-sectional survey, or a trial. The result is the number needed in each group.

Before you start you need the proportion you expect in each group (the gap between them is the smallest difference worth detecting), a confidence level (which fixes α, the false-positive rate you will accept), and a power (the probability of detecting the difference if it exists).

Equation 2.5. Sample size per group to compare two proportions
\[ {\color{#0B7B6B}{n}} = \frac{\left[\, {\color{#C2410C}{Z_{\alpha}}} \sqrt{2\, {\color{#BE185D}{\bar{p}\, \bar{q}}}} \;-\; {\color{#6D28D9}{Z_{\beta}}} \sqrt{{\color{#1D4ED8}{p_1 q_1}} + {\color{#047857}{p_2 q_2}}} \,\right]^{2}}{\left( {\color{#1D4ED8}{p_1}} - {\color{#047857}{p_2}} \right)^{2}} \]
sample size per group = [ confidence multiplier × √(2 × pooled proportion × its complement) − power multiplier × √(group 1 variance + group 2 variance) ]² ÷ (proportion in group 1 − proportion in group 2)²
Every term, defined
  • nRequired sample size per group. Double it for the whole study.
  • ZαConfidence multiplier, which sets α (the Type I error rate).1.96 for α = 0.05, two-sided.
  • ZβPower multiplier, which sets β (the Type II error rate). Written with its negative sign, so subtracting it adds to the bracket.−0.84 for 80% power (−1.28 for 90%, −1.645 for 95%).
  • p1, q1Expected proportion with the outcome in group 1, and its complement q1 = 1 − p1.Diarrhea in households without cisterns: p₁ = 0.15.
  • p2, q2Expected proportion in group 2, and its complement. The difference p1 − p2 in the denominator is the effect you want to detect; smaller differences need far more people.Households with cisterns: p₂ = 0.10.
  • p̄, q̄Pooled proportion, the average of the two groups: p̄ = (p1 + p2) ÷ 2, and q̄ = 1 − p̄. It describes the variability when the two groups are assumed alike (the null hypothesis).(0.15 + 0.10) ÷ 2 = 0.125.
Calculator
As a decimal. q₁ is calculated for you.
As a decimal. q₂, p̄ and q̄ are calculated for you.
Required per group–

Compare two means

Objective. Test whether the average of a continuous measurement differs between two groups, with enough people per group to detect a difference of a stated size. This is the formula for an analytic study with a measured outcome, such as a trial comparing mean blood pressure between two arms. The result is the number needed in each group.

Before you start you need the standard deviation of the measurement (assumed to be the same in both groups), the smallest difference between the two means that would matter, a confidence level, and a power.

Equation 2.6. Sample size per group to compare two means
\[ {\color{#0B7B6B}{n}} = \frac{2 \left( {\color{#C2410C}{Z_{\alpha}}} - {\color{#6D28D9}{Z_{\beta}}} \right)^{2} {\color{#1D4ED8}{\sigma^{2}}}}{\left( {\color{#BE185D}{\mu_1 - \mu_2}} \right)^{2}} \]
sample size per group = 2 × (confidence multiplier − power multiplier)² × population variance ÷ (difference in means to detect)²
Every term, defined
  • nRequired sample size per group. The leading 2 is there because two groups each contribute sampling variability to the comparison.
  • ZαConfidence multiplier, which sets α (the Type I error rate).1.96 for α = 0.05, two-sided.
  • ZβPower multiplier, which sets β (the Type II error rate). Written with its negative sign, so (Zα − Zβ) = 1.96 + 0.84 = 2.80 at 80% power.−0.84 for 80% power (−1.28 for 90%, −1.645 for 95%).
  • σ²Population variance of the measurement: the standard deviation, squared, assumed equal in both groups.SD 14 mmHg: σ² = 196.
  • μ1 − μ2The difference in means you want to be able to detect, in the units of the measurement. Halving it quadruples n.A 5 mmHg difference in systolic blood pressure.
Calculator
Enter σ; it is squared for you.
Same units as the measurement.
Required per group–

Finite population correction (FPC) adjustment

Objective. Reduce a sample size you have already calculated when the population you are sampling from is small enough that your sample will be a sizeable share of it (more than about 10%). Once you have measured a large fraction of a finite population, little of it is left to be uncertain about, so fewer people are needed for the same precision. Use it only for descriptive estimates from simple or stratified random samples; it does not apply to multistage or cluster designs.

Before you start you need the sample size from one of the formulae above and the size of the population you will sample from (the number of units in your sampling frame).

Equation 2.7. Sample size corrected for a finite population
\[ {\color{#0B7B6B}{n'}} = \frac{1}{\dfrac{1}{{\color{#A16207}{n}}} + \dfrac{1}{{\color{#047857}{N}}}} \]
corrected sample size = 1 ÷ ( 1 ÷ starting sample size + 1 ÷ population size )
Every term, defined
  • n′Corrected sample size. It is always smaller than n, and the smaller the population, the bigger the reduction.
  • nStarting sample size: the number from the proportion or mean formula, calculated as if the population were infinite.196 from the proportion example.
  • NPopulation size: how many people (or units) are in the population you are sampling from. The ratio n ÷ N is the sampling fraction.A workplace of 1,000 employees: N = 1,000.
Calculator
From the proportion or mean calculator.
Units in the sampling frame.
Corrected sample size–

Clustering adjustment

Objective. Inflate a sample size you have already calculated when the design samples clusters (households, schools, villages, clinics) and takes several people from each. People in the same cluster tend to resemble one another, so each additional person within a cluster adds less new information than an independent draw. Multiplying by the design effect gives the number needed for the clustered study to reach the same precision or power as a simple random sample.

Before you start you need the sample size from one of the formulae above, an estimate of how similar people within a cluster are (the intracluster correlation, from earlier studies of the same outcome and cluster type), and the average number of people you will sample per cluster.

Equation 2.8. Sample size adjusted for clustering
\[ {\color{#0B7B6B}{n'}} = {\color{#A16207}{n}} \times \bigl[\, 1 + {\color{#1D4ED8}{\rho}} \, ( {\color{#047857}{m}} - 1 ) \,\bigr] \]
adjusted sample size = starting sample size × [ 1 + intracluster correlation × (people per cluster − 1) ]
Every term, defined
  • n′Cluster-adjusted sample size. If the starting n was per group, this is also per group.
  • nStarting sample size: the number from one of the formulae above, calculated as if every person were sampled independently.685 per group from the two-proportion example.
  • ρIntracluster correlation coefficient (rho): how alike people in the same cluster are for this outcome, from 0 (no more alike than strangers) to 1 (identical). Community surveys often see 0.01 to 0.05; outcomes shared within households can be much higher.Diarrhea within households: ρ = 0.45.
  • mAverage number of people sampled per cluster. The bracket 1 + ρ(m − 1) is the design effect (deff): with m = 1 there is no clustering and no inflation.Average household size: m = 6.
Calculator
From any of the four formulae above.
Between 0 and 1.
Average cluster size in the sample.
Adjusted sample size–

Interactive 2.4. Sample size calculator for estimating a proportion or a mean and for comparing two proportions or two means, with adjustments for a small population and for clustered sampling.

The descriptive formulae are the ones to know by heart. To estimate a proportion, n = Zα² pq / L² (Equation 2.3), and to estimate a mean, n = Zα² σ² / L² (Equation 2.4). Write p and L as decimals (15% is 0.15, and ±5 percentage points is 0.05), and use p = 0.5 when you have no idea of the proportion, because it gives the largest n. For example, with no prior estimate of the proportion and a margin of ±5 percentage points at 95% confidence, n = 1.96² × 0.5 × 0.5 / 0.05² = 384.2, which you round up to 385. The usual slips are forgetting to square Zα or L, using the range in place of σ, and rounding down; and because L and the difference to be detected are squared in the denominator, halving either roughly quadruples the required sample size.

The comparison formulae are used for a power calculation, and they give the number needed in each group, so the result is doubled for the whole study. To compare two proportions, n = [Zα √(2p̄q̄) − Zβ √(p1q1 + p2q2)]² / (p1 − p2)² (Equation 2.5), where p1 and p2 are the proportions expected in the two groups, q = 1 − p in each group, and p̄ = (p1 + p2) ÷ 2 is their average, with q̄ = 1 − p̄. To compare two means, n = 2(Zα − Zβ)² σ² / (μ1 − μ2)² (Equation 2.6), where σ is the standard deviation of the measurement, assumed to be the same in both groups, and μ1 − μ2 is the difference in means to be detected. In both formulae Zα = 1.96 for 95% confidence and Zβ = −0.84 for 80% power. Zβ is written with its negative sign, so subtracting it adds to the result: in the two-means formula, Zα − Zβ = 1.96 − (−0.84) = 2.80. For example, to detect a difference of 5 mmHg in mean systolic blood pressure when the standard deviation is 14 mmHg, n = 2 × 2.80² × 14² / 5² = 2 × 7.84 × 196 / 25 = 122.9, which is rounded up to 123 per group, or 246 in all. Worked Example 2.4 in 8.4.1 works through the two-proportion formula step by step.

Each formula returns the minimum number of people with usable data, calculated as if every person were sampled independently from a very large population. Subsection 8.4 adjusts that number for the realities of fieldwork.

8.4 Adjusting the Sample Size

A formula's answer usually has to be adjusted before it becomes a recruitment target. Three adjustments are common, and they are applied in a fixed order: calculate n from the formula and round up, multiply by the design effect if the sample is clustered, apply the finite population correction if a large share of a small population is sampled, and finally divide by the expected response proportion.

8.4.1 Clustering and the Design Effect

When the design samples clusters (households, schools, villages, clinics) and takes several people from each, people in the same cluster tend to resemble one another, so each additional person within a cluster adds less new information than an independent draw. The sample size is therefore multiplied by the design effect of 7.4.2, which for planning purposes is 1 + ρ(m − 1), the bracket in Equation 2.8, where ρ is the intracluster correlation and m the average number of people sampled per cluster. The intracluster correlation measures how alike people in the same cluster are on the outcome, on a scale from 0 to 1. By this we mean that a ρ of 0 describes clusters whose members are no more alike than strangers, so clustering costs nothing and the design effect is 1, while a ρ close to 1 describes clusters in which everyone gives nearly the same answer, so that interviewing a second member adds almost nothing. The formula also shows why the size of the clusters matters. Because ρ is multiplied by m − 1, the number of other people sampled alongside each person, even a modest correlation can produce a large design effect when many people are taken from each cluster. For example, with ρ = 0.05 and 21 people per cluster, the design effect is 1 + 0.05 × 20 = 2, which doubles the required sample. Worked Example 2.4 applies the two-proportion formula (Equation 2.5) to the cistern and diarrhea example and then adjusts the result for clustering within households; watch how large the multiplier becomes when the outcome is strongly shared within households.

Worked Example 2.4: Comparing Two Proportions

Suppose you want to determine if rainwater cisterns reduce the monthly risk of diarrhea from 15% to 10%. With 95% confidence and 80% power:

p1 = 0.15, p2 = 0.10, p = 0.125, q = 0.875

Applying Equation 2.5 with Zα = 1.96 and Zβ = −0.84 takes four steps. First, 2p̄q̄ = 2 × 0.125 × 0.875 = 0.21875, whose square root is 0.46771, so the first term is 1.96 × 0.46771 = 0.91671. Second, p1q1 + p2q2 = 0.15 × 0.85 + 0.10 × 0.90 = 0.1275 + 0.09 = 0.2175, whose square root is 0.46637, so the second term is −0.84 × 0.46637 = −0.39175. Third, subtracting the second term from the first gives 0.91671 − (−0.39175) = 1.30846, and squaring gives 1.71207. Fourth, dividing by (0.15 − 0.10)² = 0.0025 gives n = 684.8, rounded up to 685 per group, so you would need 1,370 total individuals (685 with cisterns, 685 without). Enter these values in the two-proportion calculator in Interactive 2.4 to check each step.

If the outcome is clustered within households (ρ = 0.45, average household size m = 6), the clustering adjustment multiplies this by 3.25: 685 × 3.25 = 2,226.25, rounded up to 2,227 per group, more than triple the unadjusted estimate.

That multiplier, 1 + ρ(m - 1) = 3.25, is itself the design effect for this clustered design: because responses within a household are correlated, each additional person in a cluster carries less new information than an independent draw, so a larger overall sample is needed to reach the same precision.

Worked Example 2.4 showed that a design effect of 3.25 more than triples the sample, which shows how much a strongly clustered outcome costs in precision. The clustering adjustment in Interactive 2.4 reproduces this step.

8.4.2 The Finite Population Correction

The finite population correction of 7.4.3 works in the opposite direction. If a large share (more than about 10%) of a small population is sampled by simple or stratified random sampling, the corrected sample size is n′ = 1 / (1/n + 1/N) (Equation 2.7), where N is the population size. For example, n = 385 in a population of N = 2,000 becomes 322.9, rounded up to 323. As in 7.4.3, the correction applies only to descriptive estimates from simple or stratified random samples, not to multistage or cluster designs. The Small population (FPC) tab of Interactive 2.4 calculates it.

8.4.3 Non-Response

The last adjustment allows for people who will not respond. Every formula gives the number of people with usable data that the analysis needs, so the number to invite is that number divided by the expected response proportion. If a survey needs 385 completed responses and about 70% of those invited are expected to respond, it should invite 385 / 0.70 = 550 people. The number to invite is always larger than the number needed: divide by the response proportion, never multiply by it, and check the step by multiplying back (550 × 0.70 = 385).

This section has defined the two types of statistical error and power, shown how precision, expected variation, confidence, and power determine the sample size a study needs, and set out the adjustments that turn a formula's answer into a recruitment target. The key takeaways and knowledge check that follow review these points.

Key Takeaways

  • 8.1 Every hypothesis test can go wrong in two directions. A Type I error (α) is a false positive, concluding that an effect exists when it does not, and α is conventionally set at 0.05; a Type II error (β) is a false negative, missing an effect that is really there, and β is usually set at 0.20. Power, 1 − β, is the probability that a study will detect a true effect of a stated size, so a study with 80% power detects such an effect eight times in ten, and the main way to increase power is to increase the sample size. This is why a negative result from an underpowered study is inconclusive rather than reassuring: with 40% power, for instance, β = 0.60, and the study would miss a real effect of that size six times in ten. When you name the error in a scenario, remember that a Type I error needs a positive finding and a Type II error a negative one, and that failing to reject the null hypothesis never proves that it is true.
  • 8.2 Before calculating anything, decide which kind of calculation the study needs, because that follows from its aim. A descriptive aim, such as estimating one prevalence or mean in one population, calls for a precision calculation, which asks how many people are needed so that the confidence interval is no wider than a chosen half-width L, using the expected proportion (or the variance of the measurement), L, and the confidence multiplier Zα. An analytic aim, such as comparing two groups, calls for a power calculation, which asks how many people are needed for a stated probability (usually 80%) of detecting a difference of a stated size, and adds the power multiplier Zβ and the size of that difference. Each input pushes in a predictable direction: a narrower margin, more variation, a higher confidence level, more power, or a smaller difference to detect all increase the sample size. When a study has both aims, calculate both and use the larger number, and in your answer say why the aim is descriptive or analytic before you name the calculation.
  • 8.3 The sample-size formulae turn your assumptions into a number, and a careful setup prevents most mistakes. To estimate a proportion, n = Zα² pq / L², and to estimate a mean, n = Zα² σ² / L²; to compare two proportions or two means, use the power formulae in Interactive 2.4, which give the number needed in each group (double it for the whole study). Write p and L as decimals (15% is 0.15, and ±5 percentage points is 0.05), use p = 0.5 when you have no idea of the proportion because it gives the largest n, and if you know only the range that covers about 95% of values, divide it by 4 to get σ. For example, with no prior estimate of the proportion and a margin of ±5 percentage points at 95% confidence, n = 1.96² × 0.5 × 0.5 / 0.05² = 384.2, which you round up to 385. Check your work against the common slips (forgetting to square Zα or L, using the range in place of σ, or rounding down), and remember that halving the allowable error, or halving the difference to be detected, roughly quadruples the required sample size.
  • 8.4 A formula gives the minimum number of people with usable data, so the result usually has to be adjusted before it becomes a recruitment target. If the sample is clustered, multiply by the design effect, 1 + ρ(m − 1), where ρ is the intracluster correlation and m the average number sampled per cluster; in the cistern example, 685 per group × 3.25 = 2,226.25, rounded up to 2,227 per group. If a large share of a small population is sampled by simple or stratified random sampling, the finite population correction, n′ = 1 / (1/n + 1/N), reduces the number; for example, n = 385 in a population of N = 2,000 becomes 322.9, rounded up to 323. Finally, divide by the expected response proportion to find the number to invite: 385 completed responses at 70% response means inviting 385 / 0.70 = 550 people. Work in that order (calculate and round up, apply the design effect, apply the finite population correction where it fits, then divide by the response proportion), and check the last step by multiplying back (550 × 0.70 = 385), because the most common error is to multiply by the response proportion instead of dividing by it.
Knowledge Check: this section

1. A trial of 40 patients per group finds no statistically significant difference in hospital readmission between a nurse-led follow-up programme and usual care (p = 0.40). The investigators report that the study had 30% power to detect the reduction the programme was designed to achieve. A manager concludes that the programme does not work. Which response is correct?

Failing to reject the null hypothesis is not proof that it is true, especially in an underpowered study that would miss a real effect most of the time.
A Type I error is concluding there is an effect when there is none, which requires a positive finding. Missing a real effect is a Type II error.
Power is 1 − β; with power of 0.30, β is 0.70, so a real effect of the intended size would be missed seven times in ten, and the negative result is inconclusive.
A large p-value does not make a negative result reliable. With only 30% power the study was likely to miss a real effect, so it cannot show the programme does not work.
Correct answer: a possible Type II error, so the negative result is inconclusive. A Type I error (α) is concluding that there is an effect when there is none, and a Type II error (β) is failing to detect an effect that is really there; power, 1 − β, is the probability of detecting a true effect of the stated size. Here power is 0.30, so β = 1 − 0.30 = 0.70: even if the programme works as intended, a study this small would miss the effect seven times out of ten. The second option confuses the two errors (a Type I error requires a positive finding), and the first and last options treat failing to reject the null hypothesis as proof that it is true, which an underpowered study cannot provide. (Section 5, 8.1 Type I and Type II Errors and Statistical Power, Table 2.7 and the three panels; Objective 8.1)

2. A health authority plans a survey with two aims: (1) to estimate the proportion of adults in the region who have a regular family doctor, within ±3 percentage points with 95% confidence; and (2) to test whether that proportion is lower in rural than in urban areas, with 80% power to detect a difference of 10 percentage points. Which planning statement is correct?

Estimating one proportion within a margin is descriptive and calls for a precision calculation; detecting a rural and urban difference is analytic and calls for a power calculation, adding Zβ and the difference.
Both outcomes are proportions, but Aim 2 compares two groups to detect a difference, which needs a power calculation with Zβ and the size of the difference.
This reverses the two. The confidence level enters both calculations; what separates them is whether the aim estimates one quantity (precision) or detects a difference (power).
Power applies only to analytic aims that test a difference. Aim 1 estimates one proportion within a margin, which calls for a precision calculation.
Correct answer: a precision calculation for Aim 1, a power calculation for Aim 2, and the larger of the two sample sizes. The deciding information is what each aim asks. Aim 1 is descriptive (one proportion in one population, within a stated margin), so it calls for a precision calculation, n = Zα²pq ÷ L², with the expected proportion, L = 0.03, and Zα = 1.96. Aim 2 is analytic (a comparison between two groups), so it calls for a power calculation, which keeps Zα and adds the power multiplier Zβ (−0.84 for 80% power) and the difference to be detected. The third option is tempting because Aim 1 names a confidence level, but the confidence level enters both calculations; what separates them is whether the aim is to estimate one quantity or to detect a difference. (Section 5, 8.2 Precision or Power: Choosing the Calculation; Objective 8.2)

3. A health unit wants to estimate mean systolic blood pressure among adults in a community to within ±2 mmHg with 95% confidence. In a pilot study, about 95% of readings fell between 100 and 160 mmHg. Using n = Zα²σ² ÷ L², how many adults are needed?

111 forgets to square Zα: 1.96 × 225 ÷ 4. The formula uses Zα², 3.8416, giving 3.8416 × 225 ÷ 4 = 216.1, rounded up to 217.
433 forgets to square L: 3.8416 × 225 ÷ 2. With L = 2 mmHg, L² = 4.
3,458 uses the 60 mmHg range as if it were the standard deviation. About 95% of values lie within 2 SD either side, so the range spans about 4σ and σ ≈ 15 mmHg.
The pilot range spans about 4σ, so σ ≈ 60 ÷ 4 = 15 mmHg; then n = 1.96² × 15² ÷ 2² = 216.1, rounded up to 217.
Correct answer: 217. First estimate the standard deviation from the pilot range: about 95% of values lie within 2 standard deviations on either side of the mean, so the range of 160 − 100 = 60 mmHg spans about 4σ, and σ ≈ 60 ÷ 4 = 15 mmHg. Then n = 1.96² × 15² ÷ 2² = 3.8416 × 225 ÷ 4 = 216.1, which is rounded up to 217. The other options come from common slips: 111 forgets to square Zα (1.96 × 225 ÷ 4), 433 forgets to square L (3.8416 × 225 ÷ 2), and 3,458 uses the 60 mmHg range in place of the standard deviation. (Section 5, 8.2 Precision or Power, the Expected Variation in the Data panel, and 8.3 Sample-Size Formulae and Calculators, Equations 2.3 and 2.4 in Interactive 2.4; Objective 8.3)

4. A simple-random-sample calculation shows that a school survey needs 300 completed questionnaires. The survey will instead sample whole classrooms with an average of m = 21 students each, the intracluster correlation is expected to be ρ = 0.03, and about 75% of invited students are expected to take part. How many students should be invited?

400 adjusts only for non-response (300 ÷ 0.75) and forgets the design effect, which here is 1 + 0.03 × 20 = 1.6.
The design effect is 1 + 0.03 × 20 = 1.6, so 300 × 1.6 = 480 completions are needed; dividing by the response proportion gives 480 ÷ 0.75 = 640 to invite.
Adding 25% to 480 gives 600, but 600 × 0.75 = 450 completions, short of 480. The number to invite is found by dividing by 0.75.
360 multiplies 480 by 0.75 instead of dividing, which would yield far fewer than the 480 completions needed.
Correct answer: 640. Apply the adjustments in order. The design effect is 1 + ρ(m − 1) = 1 + 0.03 × 20 = 1.6, so the clustered survey needs 300 × 1.6 = 480 completed questionnaires; dividing by the expected response proportion gives 480 ÷ 0.75 = 640 students to invite, and 640 × 0.75 = 480 checks the answer. The tempting options use the right idea in the wrong way: 600 adds 25% to 480 instead of dividing by 0.75, which would yield only 600 × 0.75 = 450 completed questionnaires; 360 multiplies 480 by 0.75; and 400 forgets the design effect (300 ÷ 0.75). (Section 5, 8.4.1 Clustering and the Design Effect, Worked Example 2.4, and 8.4.3 Non-Response; Objective 8.4)

5. A health authority plans a survey to estimate the proportion of Grade 7 students who have received the HPV vaccine. Records from a neighbouring region suggest about 80%, and the estimate should lie within ±5 percentage points with 95% confidence. Using n = Zα² pq ÷ L², how many completed questionnaires would a simple random sample need?

126 forgets to square Zα (1.96 × 0.16 ÷ 0.0025). The formula uses 1.96² = 3.8416.
13 forgets to square L (3.8416 × 0.16 ÷ 0.05). With L = 0.05, L² = 0.0025.
385 uses p = 0.5, which is the safe choice only when no earlier estimate exists. Here records suggest about 80%, so pq = 0.80 × 0.20 = 0.16.
With p and L as decimals, n = 3.8416 × 0.80 × 0.20 ÷ 0.0025 = 245.9, rounded up to 246 completed questionnaires.
Correct answer: 246. Write p and L as decimals and square both Zα and L: n = 1.96² × 0.80 × 0.20 ÷ 0.05² = 3.8416 × 0.16 ÷ 0.0025 = 245.9, which is rounded up to 246. The other options come from common slips: 126 forgets to square Zα (1.96 × 0.16 ÷ 0.0025), 13 forgets to square L (3.8416 × 0.16 ÷ 0.05), and 385 uses p = 0.5, which is the safe choice only when there is no earlier estimate of the proportion. (Section 5, 8.3 Sample-Size Formulae and Calculators, Equation 2.3; Objective 8.3)

6. In the same survey, the team plans to sample whole classrooms with an average of m = 26 students each and an intracluster correlation of ρ = 0.04, which gives a design effect of 1 + 0.04 × 25 = 2.0. If the team collected only the 246 completed questionnaires calculated for a simple random sample, in whole classrooms, what would happen to the 95% confidence interval?

The design effect multiplies the variance, so a value of 2.0 doubles the variance and multiplies the standard error by √2 ≈ 1.4, widening ±5 to about ±7 percentage points.
This applies the design effect to the standard error instead of the variance. The standard error grows by √2 ≈ 1.4, so the interval is about ±7, not ±10.
The 246 came from a simple random sample formula. In whole classrooms, similar students add less new information, so the same number gives a wider interval.
Similarity within classes reduces precision: each student adds less new information, so 246 clustered students carry about as much information as 123 independent ones.
Correct answer: about ±7 percentage points instead of ±5. The information needed is what the design effect multiplies. It is the ratio of the variance under the clustered design to the variance under a simple random sample of the same size, so a design effect of 2.0 doubles the variance and multiplies the standard error by √2 ≈ 1.4, widening the planned ±5 to about ±7 percentage points. The 246 students would carry the information of about 246 ÷ 2.0 = 123 independent students. The second option is tempting, but it applies the design effect to the standard error instead of the variance, and the last option has the effect of similarity backwards: because students in a class resemble one another, each adds less new information, so precision falls. (Section 5, 8.4.1 Clustering and the Design Effect; Objective 8.4)

✦ Pass the knowledge check with 100% to continue

Final Assessment

Final Assessment

⏱ Estimated time: 30 minutes

Bringing It All Together

This lesson covered two subjects that sit at the front of the course for the same reason: everything that follows depends on having data, and on knowing what those data can and cannot say about the population they came from. Sections 1 and 2 set out the institutional and methodological vocabulary of surveillance and outbreak investigation: the four system types, the Canadian product layer, the five dimensions of data quality, the CDC 10-step framework, FIORP, and a Canadian outbreak followed step by step. The duty epidemiologist who answered the telephone at the start of the lesson was reasoning in the same way as the rest of the course, under time pressure and inside a multi-jurisdictional structure of legislation, agencies, and dashboards.

Sections 3 to 5 turned to sampling, to which the outbreak workflow already pointed. When the population at risk cannot be enumerated, an investigation samples, and the questions of these sections then apply: which population is the target and which is the source, which frame was used, which design selected the participants, what sampling error and design effect the estimates carry, and how large a sample the question requires. The population hierarchy, the probability and non-probability designs, the central limit theorem, Type I and Type II error and power, and the sample-size formulae are the means by which a sample stands in for the population from which it was drawn.

The final assessment below draws on all five sections. Its questions ask for recognition of surveillance system types from descriptions, the steps and design choices of an investigation, the definition of populations and frames, the match between a sampling design and a question, and reasoning about error, weights, design effects, and sample size.

Key Takeaways from this lesson

  1. You can now explain what public-health surveillance is, using Langmuir's working definition (the ongoing, systematic collection, analysis, interpretation, and dissemination of health-related data for public-health action), and why data count as surveillance only when the loop is closed by decisions, alerts, or programmes. You can also classify a described system as passive, active, sentinel, or syndromic and give a strength and a limitation of each type. In the exam, classify by asking who starts each report and what is reported, and remember that a legal duty to report does not make a system active.
  2. You can trace a notifiable-disease report from the clinician or laboratory through the medical health officer and the province to PHAC, explain why infections are lost at each level of the surveillance pyramid, and judge any source, including CNDSS, FluWatch, the CCDSS, the CVSD, and the BCCDC dashboards, on timeliness, completeness, representativeness, sensitivity, and predictive value positive. The most useful reminder is that passive counts are both too low and not a random subset, because severe, young, and well-served cases are more likely to be counted than mild, rural, and under-served ones, so an answer about under-reporting should always make both points.
  3. You can describe the CDC 10-step framework, check a case definition for its three parts (clinical criteria; person, place, and time; and a level of certainty), and assign the FIORP roles of PHAC, CFIA, and Health Canada. You can also read an epidemic curve, name its shape, and estimate when the exposure occurred. In the exam, count back from the peak by the average incubation period, then check that the first and last cases fall within the minimum and maximum incubation periods after the suspected exposure.
  4. You can calculate attack rates and risk ratios for each suspect food from a line list, identify the most likely vehicle, and stratify on one food to show whether another food's association is real or produced by overlap. Always divide the number ill by the size of each group before comparing, give two reasons for the vehicle (the largest risk ratio, and most cases ate it), and remember that the food whose risk ratio holds within strata of the other is the likely source.
  5. You can distinguish a census, which carries measurement error only, from a sample, which adds sampling error, and define the target population, source population, study sample, and sampling frame for a described study. You can name a group the frame would miss and distinguish internal validity (whether results are correct for the source population) from external validity (whether they generalise to the target population). In the exam, write one sentence for each layer and link any gap in the frame to external validity.
  6. You can describe the probability designs (simple random, systematic, stratified, cluster, multistage, and targeted) and the non-probability designs (judgement, convenience, purposive, and chain-referral), and name the design of a described study by asking what is selected at each stage and how. You can explain why a non-probability sample cannot support a prevalence estimate and predict the direction of its bias by asking how the people easiest to reach differ on the outcome. Remember that a stratified design samples every group, while a cluster design samples only some of them.
  7. You can explain how the standard error, σ/√n, and the central limit theorem let a sample mean be reported with an honest confidence interval, and why complex surveys need sampling weights (1 divided by the probability of selection), attention to stratification and clustering, and a design effect in the analysis. The key reminder is that the central limit theorem makes the sample mean approximately Normal, not the data themselves, and that ignoring clustering makes standard errors too small.
  8. You can distinguish Type I error, Type II error, and power, choose a precision calculation for a descriptive aim and a power calculation for a comparison, and calculate the sample size needed to estimate a proportion or a mean or to compare two groups. You can then adjust it for clustering with the design effect, for a finite population, and for non-response, documenting every assumption as you go. Work in order (calculate and round up, multiply by the design effect, apply the finite population correction where it fits, then divide by the expected response proportion), and never multiply by the response proportion.

Core Concepts Reviewed

Section 1: Langmuir's working definition of surveillance and the action loop; the five purposes of surveillance; the four system types (passive, active, sentinel, syndromic) with Canadian examples; the notifiable-disease reporting flow from clinician to medical health officer, province, PHAC, and WHO; the federal products (CNDSS, FluWatch, CCDSS, CVSD); the BC provincial layer (BCCDC dashboards and reports, Panorama); the long-running data infrastructure (vital statistics, cancer registries, DAD/NACRS, CCHS, wastewater); and the five dimensions of data quality.

Section 2: Cluster, outbreak, epidemic, and pandemic; the CDC 10-step investigation framework and epidemic-curve shapes; FIORP and its three federal partners (PHAC, CFIA, Health Canada); the 2017–18 romaine lettuce E. coli O157:H7 outbreak, whole-genome sequencing, and comparison groups, including the case-case design; attack rates, risk ratios, and stratification in a line-list analysis; and the tension between speed and accuracy, with its equity dimension.

Section 3: Census and sample; the target, source, and study population hierarchy; the sampling frame and Canadian frames; probability sampling (simple random, systematic, stratified, cluster, multistage, targeted) and the CCHS design; and non-probability sampling (judgement, convenience, purposive, snowball, respondent-driven, and time-location sampling).

Section 4: Random variables, expected value, and variance; the Bernoulli, Binomial, Poisson, Uniform, Normal, Exponential, and log-normal distributions; the sampling distribution of the mean, the standard error, and the central limit theorem; and the analysis of complex survey data with stratification, weights, clustering, the design effect, the finite population correction, and bootstrap replicate weights.

Section 5: Type I and Type II error and statistical power; the inputs to a sample-size calculation; the formulae for estimating a proportion or a mean and for comparing two proportions or two means; and the adjustments for a finite population, for clustering, and for non-response.

Final Knowledge Assessment

Complete the following 17-question assessment, which draws on all five sections of the lesson. A score of 100% is required to complete the lesson. You may retake the assessment as many times as needed.

Final Assessment: Surveillance and Sampling (17 Questions)

Question 1: On a Monday morning, the syndromic system in a health region flags a rise in emergency-department visits for vomiting among children in one town. Which sequence of events shows the surveillance system closing the loop?

This includes dissemination, but a report published months later cannot change the response to this signal. Closing the loop needs action while it can still make a difference.
Forwarding the signal to PHAC adds it to a count but stays within collection. No one analyses, interprets or acts on it locally.
Improving coding strengthens future collection, but it does nothing with this signal. The loop closes only when information returns as decisions, alerts or programmes.
The signal moves through analysis and interpretation to dissemination and action the same day, an advisory and case finding, which is what closes the loop in Langmuir's definition.
Correct answer: the epidemiologist interprets the signal and the medical health officer acts on it. Langmuir's working definition requires the ongoing, systematic collection, analysis, interpretation, and dissemination of data for public-health action, and the loop closes only when the information returns to people who act on it, as decisions, alerts, or programmes. Only the last sequence moves the signal through analysis and interpretation to action (an advisory and case finding) while the action can still make a difference. The first option is tempting because it includes dissemination, but a report published months later cannot change the response to this signal; forwarding the data or improving the coding stays within collection. (Section 1, 1.1 Defining Surveillance and Closing the Loop; Objective 1.1)

Question 2: A regional health authority sets up a system in which paramedics record the chief complaint of every 911 call on a real-time dashboard, and an algorithm triggers a review when calls for a syndrome rise above the usual baseline. Which type of surveillance is this, and what is its main limitation?

Under-reporting is a true limitation of passive surveillance, but passive reports are diagnosed cases filed by clinicians and laboratories, not real-time pre-diagnostic signals.
Chief complaints monitored in real time against a baseline are pre-diagnostic signals, which defines syndromic surveillance; its speed comes with low specificity, so many alerts are false alarms.
Lack of representativeness is a sentinel limitation, but sentinel systems are a designated network reporting cases in a standard way. Here every 911 call is monitored as a signal.
Active surveillance means public-health staff go looking for cases. Here paramedic records feed an automatic baseline alert, which is syndromic, so the paired limitation is for the wrong type.
Correct answer: syndromic surveillance, whose main limitation is false alarms. Apply the two classification questions: the reports are not diagnoses filed by clinicians or cases sought out by public-health staff, and what is monitored is a pre-diagnostic signal (the chief complaint) compared in real time against a baseline, which is the definition of syndromic surveillance. Its strength is speed, since it can flag a cluster before any laboratory confirmation, and its limitation is low specificity, so many alerts do not turn out to be real outbreaks (a low predictive value positive). Each other option pairs a true limitation with the wrong type, so the classification has to come first; passive is tempting because no one goes out to find cases, but passive reports are diagnosed cases filed by clinicians and laboratories, not real-time signals. (Section 1, 1.2 The Four Types of Surveillance System and Table 2.1; Objective 1.2)

Question 3: Suppose that for every 100 Campylobacter infections in a region, 50 cause symptoms that the person notices, 20 of those people see a clinician, 10 of them have a stool sample tested, 9 of the tests are positive, and 8 of the positive results are reported to the medical health officer and passed on to the province and PHAC. A colleague says, "The national count is low mainly because clinicians do not file their reports." Which response is correct?

Even with perfect reporting, only 9 of every 100 infections would be counted, because most are lost before a report is possible.
The reporting step loses only 1 of the 92 missing infections. Most are lost earlier, through unnoticed symptoms, no visit to a clinician and no stool test.
Following 100 infections down the pyramid loses 50, then 30, then 10 before testing is complete, 1 at testing and only 1 at reporting, leaving 8 of 100 counted.
A legal duty to report applies only to diagnosed cases. Infections lost before diagnosis never reach the count, so it understates the true number.
Correct answer: infections are lost at every step, mostly before reporting. Follow the 100 infections down the surveillance pyramid and subtract at each step: 100 − 50 = 50 lost because the illness is mild or unnoticed, 50 − 20 = 30 lost because the person does not seek care, 20 − 10 = 10 lost because no specimen is tested, 10 − 9 = 1 lost because the test misses the organism, and 9 − 8 = 1 lost because the report is never filed, leaving 8 counted. The reporting step accounts for only 1 of the 92 lost infections, so the first two options misplace the problem: even with perfect reporting, only 9 of every 100 infections would be counted. A legal duty to report applies only to cases that are diagnosed, which is why the last option is wrong. (Section 1, 2.1.2 The Surveillance Pyramid, the numbered list of places where infections are lost; Objective 2.1)

Question 4: A medical health officer needs to know, within a few days, whether COVID-19 activity in a city is rising, including among people who never seek testing. Which source best meets this need, and on which data-quality dimensions?

Wastewater results arrive within days, meeting timeliness, and do not depend on who seeks testing, so the signal is more representative of the whole affected population.
CNDSS counts only tested, laboratory-confirmed cases and can lag by weeks to months, so it fails both requirements: speed and coverage of people who never seek testing.
The CCHS does reach untested people, but it is a periodic self-report survey and cannot show this week's change, so it fails the timeliness requirement.
The CCDSS tracks chronic conditions from health-administrative data. It is not designed to monitor infection activity within days.
Correct answer: wastewater surveillance, which is strong on timeliness and representativeness. The two requirements in the stem point to two dimensions: "within a few days" is timeliness, and "including people who never seek testing" asks whether the signal reflects the whole affected population, which is representativeness. Wastewater results arrive within days and do not depend on testing behaviour, whereas CNDSS counts only tested, laboratory-confirmed cases and can lag by weeks to months. The CCHS is tempting because it reaches untested people, but it is a periodic self-report survey that cannot show this week's change, and the CCDSS tracks chronic conditions, not infections. (Section 1, 2.2.3 The Long-Running Data Infrastructure and 2.3 Five Dimensions of Data Quality; Objectives 2.2 and 2.3)

Question 5: After a banquet on a Saturday evening, illness among the guests begins on Sunday, rises to a single sharp peak on Monday, and ends by Wednesday. A second, smaller peak follows on Wednesday and Thursday, almost entirely among household members of ill guests who did not attend the banquet. The suspected organism has an average incubation period of about 2 days, with a range of 1 to 4 days. Which interpretation is best supported?

A continuous common source produces a broad plateau. Here there is one sharp peak among guests and a later wave in a different group, household members.
Among guests, onsets span the 1 to 4 day range with a single sharp peak at the 2-day average, a point source; the later peak in household members about one incubation period on fits secondary spread.
The first wave has a single sharp peak with onsets fitting the incubation period after the banquet, the point-source shape. Only the later wave fits person-to-person spread.
Nothing in the stem points to a second meal. The second wave falls about one incubation period after the first and among household contacts, the pattern of secondary spread.
Correct answer: a point source at the banquet, followed by secondary spread. The information needed is the timing of each wave relative to the banquet and to the incubation period. Among guests, the first case began 1 day after the banquet (the minimum incubation), the peak came 2 days after (the average), and the last case by 4 days after (the maximum), with a single sharp peak: the signature of a point source. The second, smaller peak came about one incubation period after the first and was confined to household members who were not at the banquet, the pattern expected when the first cases infect others. The first option is tempting because the onsets span several days, but a continuous common source produces a broad plateau, not one sharp peak followed by a later wave in a different group, and nothing in the stem points to a second meal. (Section 2, 3.3 Reading an Epidemic Curve, Figure 2.1 and the paragraph on estimating the exposure window; Objective 3.3)

Question 6: The case definition for a norovirus outbreak at a three-day conference reads: "A case is an attendee of the Harbourview conference (12 to 14 March) whose vomiting or diarrhoea began between 12 and 17 March. A case is confirmed when norovirus is detected in a stool sample, probable when no stool was tested and the person shared a hotel room with a confirmed case, and suspect when no stool was tested and there is no such link." Which person is classified correctly?

This attendee meets the clinical, person and time criteria, and a positive stool sample is the condition for a confirmed case, not a probable one.
Onset on 20 March falls outside the 12 to 17 March window, so this person is not a case at all under the definition.
The person restriction covers conference attendees only. However clearly infected, a hotel employee who did not attend is not a case under this definition.
This attendee meets the clinical criteria and the person and time restrictions, was not tested, and shared a room with a confirmed case, which is exactly the stated condition for a probable case.
Correct answer: the attendee with onset on 15 March who shared a room with a confirmed case is a probable case. Check each person against the three parts of the case definition in turn: the clinical criteria (vomiting or diarrhoea), the restrictions on person, place, and time (an attendee, with onset from 12 to 17 March), and the level of certainty. The 15 March attendee meets the clinical, person, and time criteria, was not tested, and has the stated link, so the case is probable. The attendee with a positive stool sample is confirmed, not probable; the attendee whose illness began on 20 March falls outside the time window and is not a case at all; and the hotel employee, however clearly infected, falls outside the person restriction, which covers attendees only, so this person is not counted under this definition. (Section 2, 3.2 The CDC 10-Step Investigation Framework, the paragraph on the case definition and the Step 4 panel; Objective 3.2)

Question 7: At a family reunion attended by 150 people, 36 became ill, and the investigators ran a retrospective cohort study of everyone who attended. Of the 60 guests who ate the egg salad, 27 became ill; of the 90 who did not eat it, 9 became ill. Which statement correctly gives the attack rates and the risk ratio for the egg salad, and interprets it?

Attack rates use everyone in each group as the denominator, 27 ÷ 60 = 45% and 9 ÷ 90 = 10%, and the risk ratio divides exposed by unexposed: 45% ÷ 10% = 4.5.
These divide by the 36 cases rather than by the people in each group. An attack rate's denominator is everyone in the group, eaters or non-eaters.
Subtracting gives a difference of 35 percentage points, not a ratio. A risk ratio divides the exposed attack rate by the unexposed attack rate.
This puts the unexposed attack rate on top, reversing the reference group. The exposed rate goes in the numerator, giving 4.5, a higher risk among eaters.
Correct answer: attack rates of 45% and 10%, and a risk ratio of 4.5. An attack rate is the proportion of a group who became ill during the outbreak, so its denominator is everyone in the group: 27 of the 60 egg-salad eaters (45%) and 9 of the 90 non-eaters (10%). The risk ratio divides the attack rate among the exposed by the attack rate among the unexposed, 45% ÷ 10% = 4.5, and a value above 1 means a higher risk among those who ate the food. The second option uses the wrong denominator (the 36 cases rather than the people in each group), the third subtracts instead of dividing (a difference of 35 percentage points is not a ratio), and the fourth reverses the reference group by putting the unexposed attack rate on top. (Section 2, 4.1 From Line List to Risk Ratio; Objective 4.1)

Question 8: At the same reunion, many guests who ate the egg salad also drank the lemonade, whose crude risk ratio was about 2.3. The table divides the 150 guests by both exposures. What does a stratified analysis of these groups show?

GroupAte egg saladDrank lemonadeGuestsIll
1YesYes5023
2YesNo104
3NoYes303
4NoNo606
A crude risk ratio above 2 can arise from overlap between exposures. Within strata of egg salad, the lemonade's risk ratio is about 1, so it is not an independent vehicle.
Risk ratios from small strata are rarely identical. The egg salad's stays large in both strata (4.6 and 4.0), while the lemonade's falls to about 1, so the egg salad is the likely vehicle.
Holding egg salad constant, the lemonade's risk ratio is about 1 (1.15 and 1.0), while the egg salad's stays about 4 in both lemonade strata, so the lemonade's crude association came from overlap.
Guests who had both and guests who had neither differ on two exposures at once, so that comparison cannot separate them. Stratifying shows the lemonade's risk ratio is about 1.
Correct answer: the lemonade's association came from its overlap with the egg salad (50 of the 80 lemonade drinkers also ate egg salad). The attack rates are 23 ÷ 50 = 46%, 4 ÷ 10 = 40%, 3 ÷ 30 = 10%, and 6 ÷ 60 = 10%. Holding egg salad constant, the lemonade risk ratio is 46% ÷ 40% = 1.15 among egg-salad eaters (group 1 against group 2) and 10% ÷ 10% = 1.0 among non-eaters (group 3 against group 4), close to 1 in both strata; holding lemonade constant, the egg-salad risk ratio is 46% ÷ 10% = 4.6 among lemonade drinkers and 40% ÷ 10% = 4.0 among non-drinkers, large in both. The second option is tempting because 4.6 and 4.0 differ, but risk ratios from small strata are rarely identical; what matters is that the egg salad's risk ratio stays large in both strata while the lemonade's falls to about 1. The last option compares groups that differ on both exposures at once, so it cannot separate them. (Section 2, 4.1 From Line List to Risk Ratio and Worked Example 2.1; Objective 4.1)

Question 9: A health authority wants to estimate how common asthma is among adults who work in agriculture in its region. It obtains the membership lists of the region's two farm-workers' associations, randomly samples 600 members, and 410 of them complete the survey. Which set of definitions is correct?

This inverts the hierarchy. The target population is the broadest group described, all agricultural workers; the 410 respondents are the study sample, not the target.
The target is all agricultural workers in the region, the source is the listable association members, the frame is the membership lists linking them, and the study sample is the 410 respondents.
The target population is the broadest group the estimate should describe, all agricultural workers, not association members; and the frame is the membership lists, not all farms.
The target and frame are right, but the source population contains only people who can be sampled from the frame, the association members, and the 600 become the study sample only if they take part.
Correct answer: the target population is all agricultural workers in the region, the source population is the association members, the study sample is the 410 respondents, and the frame is the two membership lists. Write one sentence for each layer. The target population is the broadest group the estimate is meant to describe (all adults working in agriculture in the region); the source population is the listable group from which participants can actually be drawn (the association members); the sampling frame is the list that links the two (the membership lists); and the study sample is the people who end up in the data (the 410 who completed the survey). The fourth option is tempting because its target population and frame are right, but the source population contains only the people who can be sampled from the frame, not all adults in the region, and the 600 people selected become the study sample only if they take part. (Section 3, 5.2.1 The Hierarchy of Populations, Figure 2.2, and 5.2.2 The Sampling Frame; Objective 5.2)

Question 10: In a survey of agricultural workers drawn from the membership lists of two farm-workers' associations, the team learns that seasonal and temporary workers are rarely association members. Which statement about this gap is correct?

Workers missing from the frame lie outside the source population; the estimate may be valid for members (internal validity) but may not generalise to all agricultural workers (external validity).
This confuses the two kinds of validity. Leaving non-members out does not make the estimate wrong for members; it limits generalisation to the target population.
Random selection protects the estimate for the people on the lists but cannot reach people who are not on them, so the gap still limits generalisation.
Sampling more members adds no one to the lists. Seasonal and temporary workers who are not members stay outside the frame however large the sample.
Correct answer: the gap threatens external validity. Internal validity concerns whether the result is correct for the source population (here, the association members), and external validity concerns whether it can be generalised from the source population to the target population (all agricultural workers in the region). Seasonal and temporary workers are not on the frame, so they are outside the source population: leaving them out does not make the estimate wrong for members, but it limits how far the estimate can be generalised, especially if their asthma risk differs. The second option is the tempting confusion of the two kinds of validity; random selection from the lists protects the estimate for members but cannot reach people who are not on the lists, and sampling more members adds no one to the lists, which is why the last two options are wrong. (Section 3, 5.2.1 The Hierarchy of Populations, the paragraph on internal and external validity, and 5.2.2 The Sampling Frame, the Canadian frames; Objective 5.2)

Question 11: To estimate how many children aged 2 to 5 in a province are up to date with their vaccinations, a team randomly selects 20 of the province's 120 licensed child-care centres and then randomly selects 15 children from each selected centre's enrolment list. Which sampling design is this?

A stratified design would sample children from every centre. Here only 20 of 120 centres are chosen at random, which makes it a cluster design.
A one-stage cluster sample takes everyone in the selected clusters. Here only 15 children are sampled within each centre, so there is a second stage.
Taking the same number from each centre does not make a design systematic. A systematic sample takes every jth unit after a random start.
Some groups (20 of 120 centres) are chosen at random as primary sampling units, then children are sampled at random within each, which is a two-stage cluster sample.
Correct answer: a two-stage (multistage) cluster sample. Ask what is selected at each stage and how: at the first stage some of the groups (20 of the 120 centres) are chosen at random, and at the second stage a random sample of individuals (15 children) is drawn within each chosen group. Choosing only some groups makes it a cluster design, and sampling within the chosen groups, rather than taking everyone in them, makes it two-stage. The first option is the classic confusion, since a stratified design would sample children from every centre, not from 20 of them; the second is wrong because only 15 children per centre are taken; and taking the same number from each centre does not make a design systematic, which requires taking every jth unit after a random start. (Section 3, 6.1.2 The Six Probability Designs, the Multistage tab, and 6.1.3 Naming the Design of a Described Study, Table 2.3 and the paragraph after it; Objective 6.1)

Question 12: To estimate the proportion of adults in a city who have a regular family doctor, a team interviews adults in the waiting rooms of family-practice clinics, where patients wait to see their own doctor, approaching every third person who arrives. Which statement is correct?

Taking every third arrival is a rule applied to clinic attenders, not a random draw from a frame of the city's adults. Adults who are not in these waiting rooms cannot be selected.
The design label is right, but the direction is not. People waiting to see their own family doctor almost all have one, so the estimate would be too high.
Selection probabilities for the city's adults are unknown, and nearly everyone waiting for their own family doctor has one while adults without one are rarely there, so the estimate is too high.
A larger convenience sample makes the estimate more precise but no less biased. The bias comes from who is reachable, which size cannot change.
Correct answer: a convenience sample that would overestimate the proportion. The selection probability of each adult in the city is unknown, because only people who happen to be in these waiting rooms can be selected; taking every third arrival is a rule applied to clinic attenders, not a random draw from a sampling frame of the city's adults, which is why the first option is wrong. To predict the direction of the bias, ask how the people easiest to reach differ on the outcome: patients waiting for their own family doctor almost all have one, while adults without a doctor are rarely there, so the estimate would be too high. A larger sample makes the estimate more precise but no less biased, which is why the last option is wrong. (Section 3, 6.2.2 Why Non-Probability Samples Cannot Estimate Prevalence, the callout and the paragraph after it; Objective 6.2)

Question 13: A school-based survey of 2,000 students in 50 schools estimates that 20% of students vape. An analyst treats the data as a simple random sample and reports a standard error of 0.9 percentage points. The design effect for this estimate is 2.25. Which statement is correct?

The design effect multiplies the variance, so the standard error grows by √2.25 = 1.5, from 0.9 to 1.35 percentage points, and the analyst's interval is too narrow.
This applies a ratio of variances directly to the standard error. The standard error grows by the square root of the design effect, 1.5, not by 2.25.
Clustering makes estimates less precise, not more. A design effect above 1 means a larger standard error than a simple random sample of the same size.
Sample size does not remove the design effect. A design effect above 1 means less precision than a simple random sample of the same size, however large the sample.
Correct answer: about 1.35 percentage points, so the analyst's interval is too narrow. The design effect is the ratio of the variance under the actual design to the variance under a simple random sample of the same size, so a design effect of 2.25 means the variance is 2.25 times larger, and the standard error, the square root of the variance, is √2.25 = 1.5 times larger: 0.9 × 1.5 = 1.35 percentage points. The 95% confidence interval therefore widens from about 20 ± 1.8 (1.96 × 0.9) to about 20 ± 2.6 (1.96 × 1.35) percentage points. Multiplying the standard error by 2.25 is the tempting error, because it applies a ratio of variances to a standard error; and a design effect above 1 always means less precision than a simple random sample of the same size, however large the sample. (Section 4, 7.4.2 The Design Effect, the paragraph on interpreting a design effect; Objective 7.4)

Question 14: A health unit wants to know whether text-message reminders raise the proportion of children who are up to date with their vaccinations, from about 75% among families who receive no reminders to 85% among families who receive them. Which sample-size calculation fits this aim, and with which inputs?

The outcome is a proportion, but the aim is to detect a difference between two groups. A precision calculation sizes a study to estimate one proportion within a margin.
A power calculation for two groups needs both expected proportions and Zβ as well as Zα. Power is chosen by the investigators, not fixed by the formula.
The two calculations answer different questions and give different sample sizes. The aim decides the calculation: detecting a difference calls for power.
Detecting a difference between two groups is an analytic aim, so a power calculation applies, using both expected proportions, Zα = 1.96 and Zβ = −0.84 for 80% power.
Correct answer: a power calculation with the two expected proportions, Zα, and Zβ. The deciding information is the aim: comparing two groups to detect a difference of a stated size (75% against 85%) is an analytic aim, which calls for a power calculation, sized so that the study has a stated probability (the power, 1 − β) of detecting that difference if it is real. The two-proportion formula therefore needs both expected proportions, the confidence multiplier Zα = 1.96, and the power multiplier Zβ = −0.84 for 80% power, and it gives the number needed in each group. The first option is tempting because the outcome is a proportion, but a precision calculation sizes a study to estimate one proportion within a margin, not to detect a difference between groups. (Section 5, 8.2 Precision or Power: Choosing the Calculation and 8.3 Sample-Size Formulae and Calculators; Objectives 8.2 and 8.3)

Question 15: A workplace of 1,000 employees plans a simple random sample to estimate the proportion who smoke, which is expected to be about 15%, to within ±5 percentage points with 95% confidence. Using n = Zα²pq ÷ L² and then the finite population correction, n′ = 1 ÷ (1/n + 1/N), how many completed responses are needed?

196 is right before the correction, but it is about a fifth of the 1,000 employees, so the finite population correction applies and reduces the number needed.
n = 3.8416 × 0.1275 ÷ 0.0025 = 195.9, rounded to 196; because that is a large share of 1,000, the correction gives 1 ÷ (1/196 + 1/1,000) = 163.9, rounded up to 164.
91 comes from forgetting to square Zα (giving 100) and then correcting. The formula uses 1.96² = 3.8416.
A census is not needed. Sampling a large share of a small population leaves less uncertainty, which is why the finite population correction lowers the sample size.
Correct answer: 164. First, n = 1.96² × 0.15 × 0.85 ÷ 0.05² = 3.8416 × 0.1275 ÷ 0.0025 = 195.9, rounded up to 196. Because 196 is about a fifth of the 1,000 employees and the design is a simple random sample with a descriptive aim, the finite population correction applies: n′ = 1 ÷ (1/196 + 1/1,000) = 1 ÷ (0.00510 + 0.00100) = 163.9, rounded up to 164. The tempting answer, 196, stops before the correction; 91 comes from forgetting to square Zα (which gives 100) and then correcting; and a census of all 1,000 is not needed, because sampling a large share of a small population leaves less uncertainty, not more. (Section 5, 8.4.2 The Finite Population Correction, Equation 2.7, and Interactive 2.4, Small population (FPC); Objective 8.4)

Question 16: During a five-day summer heat wave, a regional health authority's emergency-department triage system flags every visit with a chief complaint of heat exhaustion, heat stroke, or dehydration and raises an alert when the daily count exceeds the usual baseline. It records 140 such visits, 98 of them by adults aged 65 and over. Which statement about this system and its count is correct?

Emergency-department visits are not a random sample of heat illness. Mild cases and people with poorer access to care are under-represented, so the count cannot be scaled up directly.
Syndromic systems do raise false alarms, but this is syndromic, not sentinel, and the larger problem is the many heat illnesses that never reach an emergency department.
Triage chief complaints monitored against a baseline are syndromic; as in the surveillance pyramid, mild illness and poor access keep cases out, so the count is low and not a random subset.
Active surveillance means staff go looking for cases in the community. A triage system only sees people who reach an emergency department, so it misses much heat illness.
Correct answer: syndromic surveillance whose count is too low and not a random subset. The information needed is what is reported and how illness reaches the count. Pre-diagnostic signals (triage chief complaints) monitored in real time against a baseline define syndromic surveillance. As in the surveillance pyramid, heat illness is lost before it is counted: people with mild illness rest at home and never seek care, and people with poorer access to care, such as older adults who live alone or have limited mobility, are less likely to reach an emergency department. The losses are not random, so the visits under-represent the most vulnerable. The second option is tempting because syndromic systems do have low specificity, but the larger problem here is the many heat illnesses that never appear, and the system is syndromic, not sentinel. (Section 1, 1.2 The Four Types of Surveillance System and 2.1.2 The Surveillance Pyramid; Objectives 1.2 and 2.1)

Question 17: After the heat wave, the authority plans a survey to estimate the proportion of adults aged 65 and over living in their own homes who have a cool place at home. It will draw a simple random sample from the provincial health-insurance client registry, separately in urban and rural areas. An earlier study suggests that about 60% have a cool place; the estimate should lie within ±5 percentage points with 95% confidence, and about 75% of those invited are expected to take part. Using n = Zα² pq ÷ L², how many adults should be invited?

The descriptive aim calls for a precision calculation, n = 3.8416 × 0.24 ÷ 0.0025 = 368.8, rounded to 369; with no clusters there is no design effect, and 369 ÷ 0.75 = 492 to invite.
277 multiplies 369 by 0.75 instead of dividing. Inviting 277 would yield only about 208 completed interviews.
462 adds 25% to 369. Then 462 × 0.75 is about 347 completions, short of the 369 needed; the correct step divides by 0.75.
514 uses p = 0.5 (385 ÷ 0.75). The safe value is used only when no earlier estimate exists, and here a study suggests about 60%.
Correct answer: 492. The information needed is the aim, the expected proportion, the margin, and the response proportion. The aim is descriptive (one proportion in one population within a stated margin), so a precision calculation applies: n = 1.96² × 0.60 × 0.40 ÷ 0.05² = 3.8416 × 0.24 ÷ 0.0025 = 368.8, rounded up to 369 completed interviews. No design effect is needed, because no clusters are sampled, so the number to invite is 369 ÷ 0.75 = 492 (check: 492 × 0.75 = 369). The tempting errors are 277, which multiplies 369 by 0.75 instead of dividing; 462, which adds 25% to 369 and falls short; and 514, which uses p = 0.5 (385 ÷ 0.75) even though an earlier estimate of 60% is available. (Section 5, 8.2 Precision or Power: Choosing the Calculation, 8.3 Sample-Size Formulae and Calculators and 8.4.3 Non-Response; Objectives 8.2, 8.3 and 8.4)
✦ Pass every section knowledge check with 100% before submitting

Lesson complete

You have successfully completed this lesson: Surveillance and Sampling. Continue to Lesson 3: Questionnaire Design.

This lesson set up vocabulary that the rest of the course builds on: the four surveillance system types, the major Canadian products, the CDC 10-step outbreak framework and FIORP, a line-list analysis, the target, source, and study population hierarchy, the probability and non-probability sampling designs, the central limit theorem and the design effect, Type I and Type II error and power, and the sample-size calculations that make a sample defensible.

Choosing the people in a study is one link in the design chain: once they have been chosen, the questions to ask them and the way to ask them have to be decided.