# Lesson 2: The Rise of Public Health

*Companion-podcast transcript • Sarah & Kiffer*  
*Office Hours episode to listen to after working through the lesson*

---

**Sarah:** Welcome back to Office Hours. I'm Sarah.

**Kiffer:** And I'm Kiffer. This episode goes with Lesson two, The Rise of Public Health.

**Sarah:** It's meant for after you've worked through the lesson. We'll add some perspective on where these ideas came from and how they're used today, some critique of the tools and the institutions, a few of the questions students tend to find thorny with this material, and some extra worked examples, including a couple that are harder than the ones on the lesson page.

**Kiffer:** There are a few places in the episode where we'll ask you to work something out before we give the answer. When that happens, you'll hear a few seconds of quiet, and you're welcome to pause the audio if you'd like more time to think.

**Sarah:** We'll take four questions. First, when a case count goes up, what do you need to know before you can say a disease is spreading? Second, what made John Snow's Grand Experiment so convincing, and where could it still have gone wrong? Third, how do you judge a surveillance system when no system can be good at everything? And fourth, who should produce the numbers that compare the health of whole countries? Kiffer and I see that last one a little differently.

**Kiffer:** Question one. When a count goes up, what do you need before you can say the disease is spreading? The short answer from the lesson is a numerator, a denominator and a process. Let's make that concrete with a made-up example that has a few traps in it. Picture a fast-growing suburban health region. In 2019 it reported 160 confirmed cases of a notifiable infection, and in 2024 it reported 240. Over those five years, its population grew from 400,000 to 480,000, and the number of tests done for this infection went from 8,000 a year to 15,000.

**Sarah:** I'd like to try this one aloud, because I think it's harder than it looks.

**Kiffer:** Before you do, let's give everyone a chance. Using those numbers, is this infection more common in 2024 than it was in 2019? Work out whatever you need. Take a few seconds.

*(Pause)*

**Sarah:** Okay. The count went from 160 to 240, which is eighty more, or a fifty percent rise. But I know I need a rate. So 240 divided by 400,000, times 100,000, gives sixty per 100,000. In 2019 it was 160 divided by 400,000, which is forty per 100,000. Sixty compared with forty is still a fifty percent rise, so the rate confirms the count.

**Kiffer:** Look at what you just found, though. The rate rose by exactly the same fifty percent as the count. When can that happen?

**Sarah:** Only when the population hasn't changed. And it did change. I used the 2019 population for both years. For 2024 it should be 240 divided by 480,000, which is 0.0005, or fifty per 100,000.

**Kiffer:** Right. That's an easy slip to make, because population estimates are usually updated less often than case counts, and an out-of-date denominator makes a growing place look sicker than it is. So the rate went from forty to fifty per 100,000. What does that tell you?

**Sarah:** Fifty divided by forty is 1.25, so a twenty-five percent rise. That's smaller than the count suggested, but it's still a rise. So I'd say it's spreading, just more slowly.

**Kiffer:** That's the next trap. What about the third item on the checklist, the process that produced the count?

**Sarah:** The testing. It almost doubled, from 8,000 to 15,000, and if you test more people you find more cases. Let me work out the share of tests that came back positive. In 2019, 160 divided by 8,000 is 0.02, or two percent. In 2024, 240 divided by 15,000 is 0.016, or 1.6 percent. So the positive share actually fell.

**Kiffer:** So now the rate per population is going up and the share of positive tests is going down. Which one would you believe?

**Sarah:** My instinct is to trust the positive share, since it takes account of the extra testing.

**Kiffer:** It takes account of how many tests were done, but it also depends on who got them. If most of the extra 7,000 tests went to lower-risk people, say contacts screened without symptoms, the positive share would fall even if the infection had become more common. So neither number settles the question by itself.

**Sarah:** Then what would settle it?

**Kiffer:** A measure that doesn't depend on who decides to get tested. Hospital admissions for the infection would help, or results from a fixed set of sentinel clinics that use the same testing criteria every year, or, for some pathogens, wastewater. If those stayed flat, most of the rise in the rate is probably a testing effect. If they rose as well, the spread is more likely to be real.

**Sarah:** And a change in testing can push counts down just as easily as up. There's a Canadian example of that from the Omicron wave, isn't there?

**Kiffer:** There is. As of December 31, 2021, Ontario recommended PCR testing only for people with symptoms who belonged to specific higher-risk groups, such as hospital patients and health care workers. Most other people with symptoms were asked to stay home and isolate. Ottawa Public Health then noted on its dashboard that confirmed cases were an underestimate of the true number of people with COVID-19, and Ottawa's medical officer of health, Dr. Vera Etches, said hospitalizations would be followed as a key indicator.

**Sarah:** So from that point the case count was measuring the testing policy as much as the virus.

**Kiffer:** Yes. Whenever a count changes, it's worth asking whether the way it was produced changed at the same time.

**Sarah:** Question two takes us back to 1854 and to Snow's Grand Experiment in south London. Broad Street gets the famous image, but I find the Grand Experiment more convincing. Kiffer, why does it hold up so well?

**Kiffer:** Because the two groups were mixed together on the same streets. In 1852, the Lambeth Company moved its water works up the Thames to Thames Ditton, above the reach of London's sewage, while the Southwark and Vauxhall Company kept drawing water from the river in the city. In parts of south London, Snow wrote, the pipes of both companies went down every street, so next-door neighbours could be drinking very different water.

**Sarah:** And Snow said many residents couldn't name the company that supplied them, which sounds like a recipe for misclassifying the exposure.

**Kiffer:** That's one of my favourite details in the whole story. Snow found he could tell the two waters apart with a chemical test, because the Southwark and Vauxhall water contained far more salt. So he had a check on the exposure that didn't rely on what people remembered about their water bills.

**Sarah:** Let's do his numbers, because they're real and they're in his book. In Table nine, for the first seven weeks of the 1854 epidemic, houses supplied by Southwark and Vauxhall numbered 40,046, with 1,263 cholera deaths. Houses supplied by Lambeth numbered 26,107, with 98 deaths.

**Kiffer:** Snow turned each of those into deaths per 10,000 houses. For Southwark and Vauxhall, 1,263 divided by 40,046, times 10,000, comes to about 315. For Lambeth, 98 divided by 26,107, times 10,000, comes to about 37. And 315 divided by 37 is about 8.5, which is why Snow wrote that the mortality was between eight and nine times as great.

**Sarah:** Before we celebrate, here's a question for everyone. What could still make this comparison unfair? Think about the denominator, and about who lived in each group. Take a few seconds.

*(Pause)*

**Kiffer:** The denominator is houses, and a house can hold very different numbers of people. If Southwark and Vauxhall houses held more people on average, a higher death count per house would partly reflect more people per house. The house counts also came from a return made to Parliament for 1853, a year earlier than the deaths, so they're a slightly dated denominator, the same problem Sarah ran into in our first example.

**Sarah:** And the groups might have differed in other ways, like poverty or how low-lying their streets were. That was exactly what the miasma side would have argued, since low ground was supposed to carry bad air.

**Kiffer:** Right, and William Farr had found in the 1849 epidemic that cholera deaths fell as elevation rose, which he read as support for the miasma view. The mixing of the two water supplies on the same streets is what answers that objection. Poverty and elevation were roughly shared between neighbours, and the water was not. That's what makes it a natural experiment.

**Sarah:** And Farr ends up as part of the story, too, doesn't he?

**Kiffer:** He does. Snow passed his early results to Farr, and at Farr's suggestion the registrars in south London were asked to record the water supply of the house in every cholera death after the 26th of August. By the 1866 epidemic, Farr had accepted Snow's explanation. I like that as a picture of how the field works. A skeptic who runs the data system can still help test the idea he doubts.

**Sarah:** Question three is about surveillance systems. The lesson lists the attributes from the CDC's 2001 guidelines, things like simplicity, acceptability, sensitivity, predictive value positive, representativeness and timeliness. Students tend to memorize the list, so let's use two of them with numbers.

**Kiffer:** Here's a made-up syndromic system. A health region watches emergency department visits for vomiting and diarrhea, and the system raises an alert whenever visits climb well above the usual level for that time of year. Over three years, it raises forty alerts. Investigators follow up every one, and six turn out to be real outbreaks. Over the same three years, the region's lab reports and other investigations identified eight outbreaks of the kind the system was meant to catch.

**Sarah:** So here's the question for everyone. What is this system's sensitivity, and what is its predictive value positive? Take a few seconds.

*(Pause)*

**Sarah:** Sensitivity is the share of real outbreaks that the system caught. It caught six of the eight, which is seventy-five percent. Predictive value positive is the share of alerts that turned out to be real, so six of forty, which is fifteen percent.

**Kiffer:** So roughly one alert in seven is real. Is that a bad system?

**Sarah:** It depends on what each false alarm costs. If an alert means an epidemiologist spends an hour checking the data, fifteen percent might be perfectly acceptable. If every alert means sending inspectors out to restaurants and schools, it gets expensive quickly.

**Kiffer:** Now suppose the region raises the threshold, so an alert needs a bigger jump in visits. Over the next made-up three years, the system raises twelve alerts, and five of them are real, out of eight outbreaks again.

**Sarah:** Then sensitivity falls to five of eight, which is 62.5 percent, and predictive value positive rises to five of twelve, which is about forty-two percent.

**Kiffer:** That's the trade-off from the lesson, in numbers. A stricter threshold buys fewer false alarms and pays for them in missed outbreaks. Timeliness often suffers as well, because a bigger jump takes longer to build up.

**Sarah:** Something bothers me about the sensitivity, though. We only know about the eight outbreaks because some other system found them. What about the ones nobody found?

**Kiffer:** That's a real limitation. Sensitivity is measured against an imperfect reference. If both systems tend to miss the same kinds of outbreaks, for example small ones in communities with poor access to an emergency department, the system will look more sensitive than it is. It will also be less representative than it looks, because the people it misses aren't random.

**Sarah:** Which connects to what COVID-19 showed. Case counting performed unevenly because case definitions and testing varied so much between jurisdictions, and wastewater became useful partly because it doesn't depend on who chooses to get a test.

**Kiffer:** Wastewater has its own blind spots, though. It only covers homes connected to the sewer system that's being sampled, so households on septic tanks, and many rural and remote communities, aren't in the signal. Every system has a population it can see and a population it can't, and part of evaluating it is naming both.

**Sarah:** Question four is about the Global Burden of Disease Study and the DALY. Let's start with a calculation, because the DALY only makes sense once you've built one.

**Kiffer:** Here's a made-up region with two conditions. Condition A is a cancer. It causes forty deaths a year, and on average each death comes twenty years short of the reference life expectancy. About three hundred people are living with it at any time, with a made-up disability weight of 0.3. Condition B is a chronic pain condition. It causes no deaths, about thirty thousand people are living with it, and its made-up disability weight is 0.04. Which condition accounts for more DALYs in a year? Take a few seconds.

*(Pause)*

**Sarah:** For Condition A, the years of life lost are forty deaths times twenty years, which is eight hundred. For the years lived with disability over one year, each person living with it adds one year times the weight, so three hundred times 0.3 is ninety. That's eight hundred and ninety DALYs.

**Kiffer:** And what about Condition B, the chronic pain condition with no deaths at all?

**Sarah:** No years of life lost. Thirty thousand people times one year times 0.04 is twelve hundred. So Condition B has more DALYs, about a third more than Condition A, even though a ranking by deaths would leave it off the list entirely.

**Kiffer:** That's the reason the GBD uses DALYs. Now notice how much depends on the weight. If Condition B's weight were 0.02, half of what we used, its total would drop to six hundred, and Condition A would come first. A small number that most readers never see decides the ranking.

**Sarah:** Which brings me to the part I'm uneasy about. The GBD is produced by one academic institute, the Institute for Health Metrics and Evaluation at the University of Washington, and its estimates are used by the WHO, the World Bank and national ministries. National statistical offices sometimes disagree with its numbers for their own countries. That's a lot of measurement authority in one place.

**Kiffer:** There's a well-known case of a disagreement like that, although it was between the institute and the WHO. In 2012, a team led by Christopher Murray at the institute estimated that 1.24 million people died of malaria in 2010. The WHO's estimate for the same year was 655,000. Most of the gap was in people over age five, where the study found about 433,000 more deaths than the WHO did. Many of the countries involved have no death certificates for most deaths, so the estimates leaned heavily on verbal autopsies, which are interviews with families about a person's symptoms before death.

**Sarah:** To me that makes my point. Two expert groups came up with figures nearly a factor of two apart, and many of the countries concerned had no registration data of their own that could settle it. I'd want national vital registration, built the way Farr built it, to be the foundation, with the GBD as a way to fill gaps until those systems exist.

**Kiffer:** I read the same case as support for having the GBD. Without an independent estimate, the official figure for adult malaria deaths might never have been challenged. And even countries with complete registration code causes of death differently, so a common framework, with the same case definitions and the same reference life expectancy for everyone, is the most practical way to compare them. I'd keep the GBD and want more than one group producing estimates, so they can check each other.

**Sarah:** I agree that registration takes decades to build and that comparison needs a common framework. My concern is more about which estimate ends up in a ministry's planning document, and whether the country had a say in producing it.

**Kiffer:** That's fair, and I'd share that concern. It's part of what the lesson means by counting being a political act as well as a technical one. Choosing the same reference life expectancy for every country is a value choice, and so is deciding whose estimate is treated as official.

**Sarah:** So here's where we land together. When you use a GBD figure, read it with its uncertainty interval, check whether a national or WHO estimate exists for the same thing, and if the two differ, find out which data sources and assumptions drive the gap before you pick one.

**Kiffer:** Let's pull it together with three things to take away.

**Sarah:** First, a count needs a numerator, a current denominator and a process. Check that the population figure matches the year of the cases, and ask whether testing or reporting changed before you say a disease is spreading.

**Kiffer:** Second, a comparison is only as fair as the groups being compared. Snow's Grand Experiment was persuasive because neighbours on the same streets had different water and shared nearly everything else.

**Sarah:** And third, every surveillance system and every summary measure rests on choices. A threshold trades sensitivity for predictive value, and a DALY ranking can turn on a disability weight, so look for those choices before you trust the number.

**Kiffer:** If you'd like more practice, rework the suburban health region example from question one with your own population and testing numbers, and each time decide which measure you would trust and what extra data would settle it.

**Sarah:** Next time, it's Lesson three, Infectious Disease, Sanitation, and Hygiene, where we pick up the story of how water, sewers and vaccines changed the health of whole populations.

**Kiffer:** Take care, everyone.

**Sarah:** See you in Lesson three.
