KnowledgeInSight
What Mathematicians Do
0% of Course 4 complete

Module 2 · Lesson 1

Statistics: From mortality tables to inference: learning about a population from a sample

You'll see how statistics grew from counting deaths in 17th-century London into a way of learning about millions of people from a few thousand. You'll work out a poll's margin of error and see why sample size matters less than how the sample was chosen.

What you will be able to do

  • Explain how a sample is used to estimate a population value and why every estimate carries uncertainty.

0% of this lesson · 9 items · 1h 3m total · 48m without the optional problem set

Contents of this lesson9 items
  1. ReadingHow Can 1,000 People Speak for a Whole Country?3 min
  2. ReadingFrom London's Bills of Mortality to Modern Statistics4 min
  3. ReadingSamples, Populations, and Bias4 min
  4. ReadingEstimation and the Margin of Error4 min
  5. ReadingThe Central Limit Theorem: Why Averages Behave Predictably4 min
  6. Guided ReadingWorked Example: Reading a Poll's Margin of Error7 min
  7. Guided ConversationJudge the Sample Behind a Claim12 min
  8. Problem Set · optionalSamples and Margins15 min
  9. Knowledge CheckFrom mortality tables to inference: learning about a population from a sample10 min

Report an issue on this page

Reading 3 min Done

How Can 1,000 People Speak for a Whole Country?

You've seen the line at the bottom of a news poll. A survey of about 1,000 adults finds that 52 percent favor some proposal, "with a margin of error of plus or minus 3 percentage points." A country may have tens or hundreds of millions of adults. The pollster talked to a tiny fraction of them, and still claims to know the national figure to within a few points.

Before reading on, try to answer two questions for yourself:

  • Why should 1,000 interviews be enough to say anything about millions of people?
  • If one country had ten times as many adults as another, would its pollsters need ten times as many interviews to get the same precision?

Most people's instinct says the answer to the second question is yes. A bigger country seems to need a bigger sample. That instinct is wrong, and seeing why is the starting point of statistics.

Think of tasting soup. If the pot is well stirred, one spoonful tells you how salty it is, whether the pot holds one liter or fifty. What matters is the stirring, not the size of the pot. In polling, the stirring is random selection. If every adult has a known chance of being picked, and the picks don't favor any kind of person, then 1,000 answers behave like 1,000 independent draws from the whole population. How close those draws come to the true national figure depends on how many draws you make. It hardly depends at all on how many people you didn't ask, as long as the population is much larger than the sample.

That is why the standard formula for a poll's margin of error contains the sample size but not the population size (Illowsky and Dean 2023, sec. 8.3). For a sample of about 1,000, the margin at the usual 95 percent level comes out near 3 points, whether the country has 5 million adults or 300 million.

The same reasoning shows where the guarantee stops. Random selection is the stirring. If the people who answer differ in some systematic way from those who don't, a bigger spoonful from the wrong part of the pot doesn't help. The margin of error describes only the luck of the draw. It says nothing about a sample that was chosen badly.

This is what statistics offers. It can't tell you the exact value of something you haven't fully measured. It can tell you how wrong an estimate is likely to be, and why. That turns a single number from a sample into a statement you can weigh: a best guess, a range around it, and a clear account of what could still make it misleading.

References

Reading 4 min Done

From London's Bills of Mortality to Modern Statistics

Introduction

Statistics is often taught as a set of formulas, as if it arrived complete. It was built over three centuries by people working on death records, planetary orbits, heredity, and farm experiments, and some of them held views that are now widely condemned. This reading follows that history from John Graunt's study of London's death records to the testing methods used today.

Counting the Dead in 17th-Century London

From 1592, London's Company of Parish Clerks published weekly Bills of Mortality: lists of the number of deaths in the city and their reported causes (Swetz 2019). John Graunt, a London merchant, collected decades of these bills. In 1662 he published Natural and Political Observations … Made upon the Bills of Mortality (Swetz 2019).

Graunt's move was to treat thousands of individual deaths as data that showed patterns. He sorted deaths by cause. He noticed that more boys than girls were born, and that higher male mortality roughly evened out the numbers. He found that death rates were higher in the city than in the countryside. He also built an early life table, estimating how many people out of a group would survive to each age (Encyclopaedia Britannica n.d.). His work is widely treated as a starting point for demography, the statistical study of human populations, and for statistics, the study of how to collect, summarize, and draw conclusions from data (Swetz 2019; Encyclopaedia Britannica n.d.).

Errors, Orbits, and Least Squares

Around 1800 the problem shifted to astronomy. Astronomers had many measurements of the same orbit, each slightly off. How should they combine them? Adrien-Marie Legendre published the method of least squares in a book on comet orbits. It chooses the curve that makes the sum of the squared errors as small as possible. Carl Friedrich Gauss published his own version in 1809 and claimed priority, saying he had used it first. Legendre fought for years to have his own priority recognized (O'Connor and Robertson 1999a).

Pierre-Simon Laplace, meanwhile, built a theory of how measurement errors behave. His 1812 treatise on probability included a treatment of least squares (O'Connor and Robertson 1999b). Work like this was part of a long move from describing data to inference: drawing conclusions, with stated uncertainty, about something you can't observe directly.

Heredity, Correlation, and Eugenics

In the late 1800s, Francis Galton studied how traits pass from parents to children. He measured how strongly two quantities vary together, an idea now called correlation. In experiments with sweet peas, he saw that the offspring of unusually large seeds tended to be closer to the average size. He named this regression, a word that now also covers methods for predicting one quantity from another (O'Connor and Robertson 2003a). Karl Pearson developed the correlation coefficient and regression analysis into general mathematical tools (O'Connor and Robertson 2003c).

Both men were prominent advocates of eugenics, the movement to "improve" human populations by controlling who had children. Galton has been described as its founder, and Pearson was the first Galton Professor of Eugenics at University College London, from 1911 to 1933 (O'Connor and Robertson 2003a; 2003c). After an inquiry into its history of eugenics, University College London removed their names from its spaces and buildings in 2020 (University College London 2020).

Testing and Estimation in the 20th Century

Working at the Rothamsted agricultural research station, Ronald Fisher introduced randomization in experiments and the analysis of variance, and in 1921 he introduced the concept of likelihood, the basis of the method of maximum likelihood. All three ideas are still in daily use. Fisher also promoted eugenics, and in 1933 he succeeded Karl Pearson in the Galton chair. In 2020 Gonville and Caius College, Cambridge, removed a window commemorating him, and University College London renamed a laboratory named for him (O'Connor and Robertson 2003d).

Jerzy Neyman and Egon Pearson, Karl Pearson's son, wrote a series of papers on hypothesis testing between 1928 and 1933, and Neyman developed the theory of confidence intervals in the 1930s (O'Connor and Robertson 2003b). Relations between Fisher and Neyman, who worked in the same building, broke down.

Conclusion

Statistics has no single founder. Its methods came from demographers, astronomers, biologists, and agricultural scientists, and several of its central figures used those methods in the service of eugenics. Historians and institutions still weigh how to acknowledge both the tools and their makers' commitments.

Key Terms

  • Bills of Mortality: Weekly lists of the number and reported causes of deaths in London, published from 1592.
  • Demography: The statistical study of human populations, including births, deaths, and survival.
  • Statistics: The study of how to collect, summarize, and draw conclusions from data.
  • Inference: Drawing conclusions, with stated uncertainty, about something that can't be observed directly.
  • Correlation: A measure of how strongly two quantities vary together.
  • Regression: Galton's name for the tendency of offspring of extreme parents to be closer to the average, now also used for methods that predict one quantity from another.

References

  • Encyclopaedia Britannica. n.d. "John Graunt." Written and fact-checked by the Editors of Encyclopaedia Britannica.
  • O'Connor, J. J., and E. F. Robertson. 1999a. "Adrien-Marie Legendre." MacTutor History of Mathematics Archive, University of St Andrews. Last updated January 1999.
  • O'Connor, J. J., and E. F. Robertson. 1999b. "Pierre-Simon Laplace." MacTutor History of Mathematics Archive, University of St Andrews. Last updated January 1999.
  • O'Connor, J. J., and E. F. Robertson. 2003a. "Francis Galton." MacTutor History of Mathematics Archive, University of St Andrews. Last updated October 2003.
  • O'Connor, J. J., and E. F. Robertson. 2003b. "Jerzy Neyman." MacTutor History of Mathematics Archive, University of St Andrews. Last updated October 2003.
  • O'Connor, J. J., and E. F. Robertson. 2003c. "Karl Pearson." MacTutor History of Mathematics Archive, University of St Andrews. Last updated October 2003.
  • O'Connor, J. J., and E. F. Robertson. 2003d. "Ronald Aylmer Fisher." MacTutor History of Mathematics Archive, University of St Andrews. Last updated October 2003.
  • Swetz, Frank J. 2019. "Mathematical Treasure: Graunt's Early Statistics on Mortality." Convergence, Mathematical Association of America, August 2019.
  • University College London. 2020. "Inquiry into the History of Eugenics at UCL." Report published February 28, 2020.

Reading 4 min Done

Samples, Populations, and Bias

Introduction

A restaurant's online rating, a customer survey, and a national poll all make the same kind of claim: a few answers stand for many. Whether that claim holds depends less on how many answers there are than on how they were gathered. This reading covers populations and samples, random sampling, the main sources of bias, and why two honest random samples still give different answers.

Population, Sample, and the Value You Want

The population is the whole group you want to know about: every adult in a country, every battery from a factory's production run, every patient with a given condition. Usually you can't measure all of it. So you measure a sample, a smaller group drawn from the population, and use it to estimate a value that describes the whole population, such as the share of adults who support a proposal or the average life of a battery.

A sample is useful only if it resembles the population in the ways that matter for the question (Illowsky and Dean 2023, sec. 1.2). The trouble is that you can't check the resemblance directly, because you don't know the population's value. That's why you measured a sample in the first place. You have to rely on how the sample was chosen.

Random Sampling

The tool statisticians rely on is the random sample: a sample chosen so that every member of the population has a known chance of selection, and chance alone decides who is picked. In the simplest version, every member has the same chance, and every possible group of the chosen size is equally likely (Illowsky and Dean 2023, sec. 1.2).

Random selection doesn't guarantee that any one sample looks like the population. It guarantees something weaker and more useful: no kind of person or item is systematically favored. Any mismatch between sample and population comes from the luck of the draw. Luck can be described with probability, and that is what makes it possible to say how far off an estimate is likely to be.

Where Bias Comes From

Sampling bias occurs when the way a sample is gathered makes some members of the population more likely to be included than others, so the sample systematically misrepresents the population (Illowsky and Dean 2023, sec. 1.2). It enters in several familiar ways:

SourceWhat happensExample
Who is easy to reachThe sample over-represents people or items that are convenient to findSurveying shoppers at one mall on a weekday morning
Who chooses to answerPeople with strong views, or more free time, respond moreOnline "vote now" polls, call-in surveys, product reviews
Who refusesPeople who decline differ from those who agreeA health survey that busy or unwell people skip
How questions are wordedThe wording pushes answers one wayAsking whether people support "protecting" versus "restricting" something

Self-selected samples, where people decide for themselves whether to take part, are often unreliable for exactly this reason (Illowsky and Dean 2023, sec. 1.2).

Bigger Isn't Better If It's Biased

A large biased sample can be worse than a small random one. Suppose a website collects 50,000 votes on a question, and the people who visit that site and bother to vote lean strongly one way. Collecting 500,000 votes the same way gives the same lean, measured more precisely. Size reduces the effect of chance. It does nothing about a sample that leans.

A random sample of 1,000 has the opposite profile. Its answer will be off by a little, in an unknown direction, but by an amount you can estimate. Statisticians generally prefer an error they can measure to one they can't.

Variability: Honest Samples Disagree

Even perfect random samples don't agree with each other. Draw two random samples of 1,000 voters from the same population and one might show 51 percent support while the other shows 53 percent. This variability, the natural variation in results from one random sample to another, is not a mistake. It's what chance selection produces (Illowsky and Dean 2023, sec. 1.2).

Variability shrinks as samples get larger, because a larger sample averages over more individual quirks. Bias doesn't shrink that way. Keeping the two apart is a basic skill in reading any statistic.

Conclusion

A sample stands for a population only through the method that chose it. Random selection removes systematic favoritism and leaves only chance variation, which can be measured and reduced by taking more observations. Bias from who is reached, who answers, and how questions are asked is not reduced by size, and has to be judged from how the data were gathered.

Key Terms

  • Population: The whole group you want to know about.
  • Sample: A smaller group drawn from the population and measured in order to estimate a value for the whole population.
  • Random sample: A sample chosen so that every member of the population has a known chance of selection and chance alone decides who is picked.
  • Sampling bias: A distortion that arises when the way a sample is gathered makes some members of the population more likely to be included than others.
  • Variability: The natural variation in results from one random sample to another.

References

Reading 4 min Done

Estimation and the Margin of Error

Introduction

"Fifty-two percent, plus or minus 3 points" packs two claims into one line: a best guess, and a statement about how far off that guess might be. This reading explains where the second claim comes from, why it shrinks so slowly as samples grow, what "95 percent confidence" means, and what the margin of error leaves out.

A Best Guess and a Range

When you measure a random sample, the result gives you an estimate: a value computed from the sample and used as a best guess for the population value. If 520 of 1,000 randomly chosen voters support a measure, the estimate is 52 percent. Statisticians call a single best guess like this a point estimate (Illowsky and Dean 2023, chap. 8).

The estimate is almost certainly not exactly right, because a different random sample would give a somewhat different number. So the estimate is reported with a margin of error: an amount added to and subtracted from the estimate to give a range of plausible values for the population. The range from the estimate minus the margin to the estimate plus the margin is a confidence interval (Illowsky and Dean 2023, chap. 8). For 52 percent with a margin of 3 points, the interval runs from 49 to 55 percent.

How Big Is the Margin?

For a percentage near 50 percent, at the usual 95 percent level, there's a quick rule of thumb. Write the sample size as n. Then the margin of error, as a proportion, is about

\frac{1}{\sqrt{n}}

Here \sqrt{n} means the square root of n, the number that multiplied by itself gives n. The rule comes from the standard formula, which multiplies a factor of about 2 by a measure of how much single answers vary, then divides by \sqrt{n}. For a split near 50–50, that measure is about 0.5, and 2 times 0.5 is 1, which leaves about 1 \div \sqrt{n} (Illowsky and Dean 2023, sec. 8.3).

Sample sizeRough margin, 1 \div \sqrt{n}
1000.10, or 10 points
4000.05, or 5 points
1,000about 0.032, or about 3 points
1,6000.025, or 2.5 points

The formula contains the sample size but not the population size. As long as the population is much larger than the sample, a poll of 1,000 has about the same margin in a small country as in a large one.

The Square Root Rule

The table shows the square root rule: the margin of error shrinks in proportion to one over the square root of the sample size. Going from 400 people to 1,600 multiplies the sample by four, but the margin only halves, from 5 points to 2.5. To halve the margin, you need four times the sample. To cut it to a third, you need nine times the sample.

This is why shrinking a margin gets expensive fast. Going from 3 points to 1.5 points means roughly quadrupling the number of interviews.

What "95 Percent Confidence" Means

The confidence level is the long-run success rate of the method that produced the interval. A 95 percent confidence level means that if you repeated the whole process many times, drawing a new random sample and computing a new interval each time, about 95 of every 100 intervals would contain the true population value (Illowsky and Dean 2023, sec. 8.3).

That's a statement about the method, not about any one interval. A particular interval either contains the true value or it doesn't. You just don't know which. "95 percent confidence" says that you used a procedure that works about 19 times out of 20.

What the Margin Leaves Out

The margin of error measures random error only: the spread you'd expect from the luck of the draw. It says nothing about bias. If the sample over-represents one group, if many people refused, or if the question was worded to push an answer, the true value can lie outside the interval no matter how small the margin is. A reported margin assumes a random sample and honest answers.

Conclusion

An estimate from a random sample comes with a margin of error that depends on the sample size, not the population size, and shrinks with the square root of that size. A 95 percent confidence interval comes from a method that captures the true value in about 95 of 100 samples. The margin covers chance alone; it doesn't cover bias.

Key Terms

  • Estimate: A value computed from a sample and used as a best guess for the population value.
  • Margin of error: An amount added to and subtracted from an estimate to give a range of plausible values for the population.
  • Confidence interval: The range from the estimate minus the margin of error to the estimate plus the margin of error.
  • Square root rule: The margin of error shrinks in proportion to one over the square root of the sample size, so halving it takes four times the sample.
  • Confidence level: The long-run share of intervals, over many repeated random samples, that would contain the true population value.

References

Reading 4 min Done

The Central Limit Theorem: Why Averages Behave Predictably

Introduction

Margins of error get attached to almost everything: vote shares, average incomes, average test scores, average waiting times. Yet these quantities come from data with very different shapes. This reading explains the result that makes one kind of reasoning work across all of them, what it requires, and where it stops.

Lopsided Data, Well-Behaved Averages

Take a population whose values are lopsided: say, the lengths of phone calls at a help line. Most calls are short, a few run very long, and a graph of call lengths has a tall peak on the left and a long tail stretching to the right. Nothing about it looks like a neat, symmetric curve.

Now take a random sample of 50 calls and compute their average length. That average is a sample mean: the average of the values in one sample. Take another random sample of 50 and compute its mean. Repeat thousands of times, and draw a graph of all those sample means.

The graph of sample means doesn't look like the graph of calls. It is symmetric and bell-shaped, centered on the true average call length for the whole population. The long calls in any one sample get averaged together with the short ones, and extreme averages turn out to be rare in both directions.

Figure: Two graphs side by side. On the left, a lopsided distribution of individual values with a tall peak at short lengths and a long tail to the right. On the right, the bell-shaped distribution of many sample means drawn from that population, centered on the population mean. Alt text: A right-skewed distribution of individual values beside a symmetric bell-shaped distribution of sample averages taken from it.

The Theorem in Words

The central limit theorem says this happens in general. For a population of almost any shape, if you take many independent random samples of the same reasonably large size, their sample means form an approximately normal distribution, the symmetric bell-shaped curve, centered on the population's true mean. The larger the samples, the closer the pattern comes to a normal distribution (Illowsky and Dean 2023, sec. 7.1).

The theorem also says how wide the bell is. Its spread, the typical distance of sample means from the center, equals the spread of individual values divided by the square root of the sample size (Illowsky and Dean 2023, sec. 7.1). One worked case: if individual call lengths typically differ from the average by 6 minutes, then means of samples of 36 calls typically differ from the true average by 6 \div \sqrt{36} = 6 \div 6 = 1 minute. Samples of 144 calls would cut that to half a minute.

Why It's the Landmark

The theorem turns an impossible task into a manageable one. You don't know the shape of the population; that's what you're trying to learn about. But you don't need to know it. Whatever the shape, the mean of a large random sample behaves in a known, bell-shaped way around the truth. The bell shape tells you how far a sample mean usually lands from the center. About 95 percent of sample means land within about two spreads of it.

That fact is what justifies margins of error. A sample percentage is a kind of mean: if you score each "yes" as 1 and each "no" as 0, the average of those scores is the share who said yes. So poll percentages, average incomes, and average waiting times can all be given margins of error by the same reasoning, and the square root of the sample size appears in each of them.

Limits

The theorem has conditions. The samples must be random and the values independent, so that one observation doesn't drag others along with it. The sample must also be large enough, and how large depends on the data. Mildly lopsided data settle into a bell shape with modest samples. Very lopsided data, with rare extreme values, need larger ones (Illowsky and Dean 2023, sec. 7.1). And the theorem is about means. It says nothing about individual values, which stay as lopsided as they were.

Conclusion

The central limit theorem says that means of large random samples cluster in a bell shape around the true mean, whatever the shape of the original data, with a spread that shrinks with the square root of the sample size. It underlies the margins of error reported for most estimates, provided the samples are random, independent, and large enough.

Key Terms

  • Sample mean: The average of the values in one sample.
  • Central limit theorem: The result that, for large independent random samples from a population of almost any shape, sample means form an approximately normal distribution centered on the population mean.
  • Normal (bell-shaped) distribution: A symmetric, bell-shaped pattern of values clustered around a center.
  • Spread: The typical distance of values from the center of their distribution; for sample means, the spread of individual values divided by the square root of the sample size.

References

Guided Reading 7 min Done

Worked Example: Reading a Poll's Margin of Error

Introduction

Poll results often arrive as a headline number and a margin of error in small print, and the two get read separately. This worked example puts them together. You'll compute a poll's margin two ways, turn it into a range, use the range to answer a yes-or-no question about majority support, and then list what the margin can't tell you.

The Problem

A poll of 1,000 randomly chosen voters finds that 52 percent support a ballot measure. Two questions follow:

  1. What range of values for support among all voters is consistent with this result, at the usual 95 percent confidence level?
  2. Does the poll show that a majority of all voters support the measure?

A few terms first. The population here is all voters; the sample is the 1,000 who were asked. The 52 percent is an estimate: a value computed from the sample and used as a best guess for the population. The margin of error is an amount added to and subtracted from the estimate to give a range of plausible population values, and that range is a confidence interval. A 95 percent confidence level means the method used to build the range captures the true value in about 95 of every 100 random samples (Illowsky and Dean 2023, sec. 8.3).

Working It Through

Step 1: Compute the rough margin

For a percentage near 50 percent, at 95 percent confidence, a quick rule gives the margin of error as a proportion. Write the sample size as n; the rough margin is 1 \div \sqrt{n}, where \sqrt{n} is the square root of n.

Here n = 1{,}000. The square root of 1,000 is about 31.6, since 31.6 \times 31.6 \approx 999. So

\frac{1}{\sqrt{1000}} \approx \frac{1}{31.6} \approx 0.032

As a percentage, that's about 3.2 points, which polls usually round to 3. That's the source of the familiar "plus or minus 3 points" for polls of about 1,000 people.

Step 2: Compute the more exact margin for 52 percent

The rule of thumb comes from a fuller formula. Write the sample proportion as p; here p = 0.52. Then 1 - p = 0.48 is the proportion who didn't say yes. The 95 percent margin of error is

1.96 \times \sqrt{\frac{p(1-p)}{n}}

The number 1.96 comes from the bell-shaped normal distribution: about 95 percent of sample results land within 1.96 spreads of the center. The square-root part is the spread of sample proportions for a sample of size n (Illowsky and Dean 2023, sec. 8.3).

Work it in pieces:

  • p(1 - p) = 0.52 \times 0.48 = 0.2496
  • Divide by n: 0.2496 \div 1000 = 0.0002496
  • Take the square root: \sqrt{0.0002496} \approx 0.0158
  • Multiply by 1.96: 1.96 \times 0.0158 \approx 0.031

The margin is about 0.031, or 3.1 percentage points. The rule of thumb gave 3.2. The two agree closely because 52 percent is near 50 percent, where the rule is built to work. For a result far from 50 percent, say 10 percent, the fuller formula gives a noticeably smaller margin than the rule of thumb.

Step 3: Write the interval

Subtract the margin from the estimate and add it:

52 - 3.1 = 48.9 \qquad 52 + 3.1 = 55.1

So the 95 percent confidence interval runs from about 48.9 to 55.1 percent. Rounded, that's about 49 to 55 percent. Any value of support in that range among all voters is consistent with what the sample showed.

Step 4: Decide whether the poll shows majority support

A majority means more than 50 percent. The question is whether the data rule out 50 percent or less.

They don't. The interval reaches down to about 49 percent, so values at and just below 50 percent are inside it. If true support were exactly 50 percent, a random sample of 1,000 could easily come back at 52 percent by the luck of the draw. So the poll doesn't show majority support. It's consistent with a majority, and it's also consistent with a tie or a narrow minority.

That doesn't make 52 percent meaningless. Values near 52 are more consistent with the data than values near the ends of the interval. But "consistent with a majority" and "shows a majority" are different claims, and only the first is supported here.

Step 5: Ask what the margin doesn't cover

The 3.1-point margin describes random error only: how much a random sample of 1,000 typically differs from the whole population by chance. It rests on the assumption that the sample was random and the answers honest. Several real sources of error fall outside it:

  • Who refused. If people who declined to answer differ in their views from those who answered, the result leans, and no margin of error captures the lean.
  • How the question was worded. "Do you support the measure to protect local water?" and "Do you support the new water fee?" can describe the same measure and get different answers.
  • Whether people changed their minds. A poll measures opinion on the days it was taken. If opinion shifts before the vote, the poll was accurate about a moment that has passed.

There are others, such as who was reachable at all and whether the poll counted only people who will actually vote. None of these shrinks with a larger sample.

Key Considerations

"Within the margin of error" doesn't mean a tie. When two numbers are within the margin, news reports often call the race "a statistical tie" or "too close to call." The accurate reading is narrower. The data can't rule out a tie, but they don't show one either. A 52 percent result still points modestly toward majority support; it just doesn't establish it.

The common mistake: treating the margin as covering every source of error. It's tempting to read "plus or minus 3 points" as "the true value is guaranteed to be within 3 points." That's a mistake for two reasons. First, the 95 percent level means the method misses about 1 time in 20, even when everything else is perfect. Second, the margin is computed only from the sample size and the result. It contains no information about refusals, wording, or timing. A poll can report a small margin and still be far off because of bias the margin never measured.

Population size doesn't appear. Notice that nothing in the calculation used the number of voters in the population. As long as the population is much larger than the sample, the margin depends on the sample size alone.

Summary

A poll of 1,000 random voters showing 52 percent support has a 95 percent margin of error of about 3.1 points, close to the rule-of-thumb 3.2. Here is the model result:

Interval: about 48.9 to 55.1 percent, or roughly 49 to 55 percent, at 95 percent confidence.

Verdict on majority support: Not shown. The interval includes 50 percent, so the data are consistent with a majority but can't rule out a tie or a narrow minority.

Errors the margin leaves out: people who refused to answer, the wording of the question, and changes of opinion after the poll was taken.

  1. The interval comes from 52 \pm 1.96 \times \sqrt{0.52 \times 0.48 \div 1000}, which is 52 \pm 3.1.
  2. The check on the verdict is whether 50 percent lies inside the interval. It does, at the lower end.
  3. The margin would not change if the same poll were run in a population ten times larger.

References

Guided Conversation 12 min Done

Judge the Sample Behind a Claim

In this conversation you'll bring a statistic you've seen recently, such as a poll, a survey result, or a product rating, and work out what it can and can't tell you. With hints when you want them, you'll judge how its sample was chosen and estimate its margin of error yourself. You'll leave with one question you'd want answered before trusting the claim.

You'll have this conversation with an AI assistant, using your own account. Choose a button to open a new chat with the prompt already filled in, then press send to start. If the chat opens empty, copy the prompt and paste it in.

Run this conversation in whichever assistant you already use:

Open in ClaudeOpen in ChatGPT
Claude desktop app

To use another LLM, simply copy and paste the prompt into its chat window.

Show the full prompt (it lists misreadings to watch for, so skip it if you would rather come to the conversation fresh) 
Guided Conversation: Judge the Sample Behind a Claim (about 12 minutes)

Note to the learner: press send to start. Everything below is facilitator guidance for the AI. It lists misconceptions to watch for, so skip it if you'd rather come to the conversation fresh.

Please facilitate a coached problem session with me. I'm an adult who is comfortable with basic high-school algebra but may be rusty or uneasy with math, exploring what statistics is and whether to study it further. I'm studying how a sample is used to learn about a population. Follow this guidance for the whole conversation.

GOAL
Explain how a sample is used to estimate a population value and why every estimate carries uncertainty, using a statistical claim I've seen myself: I judge how its sample was chosen and estimate its margin of error.

HOW TO RUN THE CONVERSATION
- Ask one question at a time, then wait for my reply. Keep each of your turns under about 120 words.
- Don't lecture. Explain a point only when I need it, then return to my claim.
- Be curious, collegial, and patient. Give hints and ask questions; don't solve the problem for me. Define terms briefly on first use (population, sample, random sample, bias, margin of error). Welcome disagreement when I give a reason.
- Coaching rules: always ask for my approach before giving any hint. Give one hint at a time, and only after I've tried. Never state an answer, such as a margin of error, before I've attempted it.
- Plain conversation only: don't search the web or create files or documents.
- Aim for about 12 minutes. Spend most of the time on topics 2 and 3. If my replies are brief, offer one concrete prompt, such as "how do you think the people in that survey ended up in it?", and move on. If I seem uncertain, shorten the conversation to 5-7 minutes. Always reach the final topic.
- Start now. Open with one or two warm sentences: this is a conversation, not a quiz; reasoning out loud matters more than being right; I can ask you to clarify anything. Then ask me to describe a statistic I've seen recently, such as a poll, a survey, or a product rating, including the number and, if I know it, how many people it was based on. If I can't think of one, offer this: "A survey of 900 randomly chosen adults finds that 40 percent exercise at least three times a week."

TOPICS, IN ORDER
1. The claim and its population. Ask what group the statistic claims to describe (the population) and who was actually measured (the sample). Follow-up: does the claim describe the sample, or does it reach for something bigger?
2. How the sample was chosen. Ask how I think the people or items got into the sample, and whether any kind was more likely to be included or to answer. Hint, only if I'm stuck: who was easy to reach, who chose to respond, how was the question worded? Follow-up: would asking ten times as many people the same way fix the problem I found?
3. A rough margin of error. Ask me to estimate how far off the number might be by chance alone, given the sample size. Ask for my approach first. Hints, one at a time: the rough 95 percent margin for a percentage near 50 percent is 1 divided by the square root of the sample size; for 900 people the square root is 30. Then ask me to write the range (estimate minus margin to estimate plus margin). If my statistic has no sample size, work with my guess. Follow-up: would this margin be any different if the population were ten times larger?
4. Closing. Ask me to state one thing I'd want to know before trusting the claim. This gives me something to take into practice with samples and margins.

CORRECT METHOD AND ANSWER
- Population: the whole group the claim is about. Sample: who was actually measured. A random sample gives every member a known chance of selection.
- Rough 95 percent margin of error for a percentage near 50 percent: 1 / sqrt(n), where n is the sample size. Examples: n = 400 gives 0.05 (5 points); n = 900 gives 1/30, about 0.033 (3.3 points); n = 1,000 gives about 0.032 (about 3 points).
- Fallback claim: 40 percent with a margin of about 3.3 points gives about 36.7 to 43.3 percent. (The fuller formula, 1.96 x sqrt(p(1-p)/n), gives about 3.2 points here.)
- To halve the margin, you need four times the sample.
- The margin depends on the sample size, not the population size, as long as the population is much larger than the sample.
- The margin of error covers random error only. It doesn't cover bias from who was reached, who answered, how questions were worded, or changes over time. A bigger sample doesn't reduce bias.

MISCONCEPTIONS TO CORRECT GENTLY
When one appears, name the accurate version briefly, then return to my claim.
- "A bigger population needs a bigger sample": for large populations, the sample size matters, not the fraction of the population sampled.
- "Within the margin of error means a tie": it means a tie can't be ruled out, not that one is shown.
- "Online polls with millions of votes are accurate": self-selected samples are biased, and size doesn't remove bias.
- "The margin of error covers all the ways a poll can be wrong": it covers chance only.

LIMITS
- Don't discuss specific current elections, candidates, or parties, even if my example is political.
- Don't introduce formulas beyond the two above.
- Write all math in plain notation.

TO FINISH
After I state what I'd want to know, close in one short turn:
- Affirm one specific thing I worked out, in my own words where possible.
- Suggest one or two next steps that fit how the conversation went. Possible steps: compare the margins for samples of 400 and 1,600; find out how a survey I've seen chose its respondents; review why a bigger sample doesn't fix bias.
- Restate my question on its own line, labeled "Before I trust it", so I can copy it.

Problem Set 15 minOptional Done

Samples and Margins

Overview

This optional problem set gives you five short problems about averages, sample size, and margins of error. For each one, first identify the population (the whole group the question is about) and the sample (the group actually measured), then compute or reason out the answer. Write margins of error in percentage points. When you're done, compare your work with the answers and methods given.

Problems

Problem 1

What is the average of 4, 6, and 8?

Once you've tried the problem, check your work against the solution.Show the solution to Problem 1Hide the solution

Answer: 6.

Method:

  1. Add the values: 4 + 6 + 8 = 18.
  2. Divide by how many values there are: 18 \div 3 = 6.
  3. Check: 6 sits in the middle of 4 and 8, which makes sense for three evenly spaced numbers.

Most common error: dividing by the largest value or by 2 instead of by the number of values. The average divides the total by the count.

Problem 2

What is the rough 95 percent margin of error for a poll of 400 people?

Once you've tried the problem, check your work against the solution.Show the solution to Problem 2Hide the solution

Answer: about 5 percentage points.

Method:

  1. The sample size is n = 400. The rough 95 percent margin of error for a percentage near 50 percent is 1 \div \sqrt{n}, where \sqrt{n} is the square root of n.
  2. The square root of 400 is 20, since 20 \times 20 = 400.
  3. So the margin is 1 \div 20 = 0.05, which is 5 percentage points.

Most common error: dividing 1 by 400 instead of by its square root, which gives 0.0025, a quarter of a point. That's far too small for a sample of 400.

Problem 3

A poll of 400 people has a margin of error of about 5 points. About how many people would a poll need for a margin of about 2.5 points?

Once you've tried the problem, check your work against the solution.Show the solution to Problem 3Hide the solution

Answer: about 1,600 people.

Method:

  1. The margin shrinks with the square root of the sample size. This is the square root rule.
  2. To halve the margin, the square root of the sample size must double. Doubling a square root means multiplying the number under it by 4.
  3. So the sample must be 4 \times 400 = 1{,}600.
  4. Check: \sqrt{1600} = 40, and 1 \div 40 = 0.025, or 2.5 points.

Most common error: doubling the sample to 800. That gives 1 \div \sqrt{800} \approx 0.035, or about 3.5 points, not 2.5.

Problem 4

A news site asks its readers to vote online on a new law, and 80 percent of 50,000 votes are "yes." What is the main problem with using this result to describe what all adults in the country think?

Once you've tried the problem, check your work against the solution.Show the solution to Problem 4Hide the solution

Answer: The sample is self-selected, not random. Only people who read that site and chose to vote are counted, so the result can't be generalized to all adults.

Method:

  1. Identify the population: all adults in the country. Identify the sample: the 50,000 readers who voted.
  2. Ask how the sample was chosen. Nobody was picked by chance. People chose themselves, and they are readers of one site who cared enough to vote.
  3. Those readers may differ in their views from adults in general, so the sample can lean in a direction that 50,000 votes, or 5 million, would never correct.

Most common error: thinking 50,000 votes must be accurate because the number is large. A large biased sample measures its own lean very precisely. No margin of error applies, because the sample isn't random.

Problem 5

A poll of randomly chosen voters reports 52 percent support for a measure, with a margin of error of 3 points. Does the poll show that a majority of all voters support the measure?

Once you've tried the problem, check your work against the solution.Show the solution to Problem 5Hide the solution

Answer: No.

Method:

  1. Write the interval: 52 - 3 = 49 and 52 + 3 = 55, so the range is 49 to 55 percent.
  2. A majority means more than 50 percent. Ask whether the interval rules out 50 percent or less.
  3. It doesn't. The interval includes 50 percent and values just below it. So the poll is consistent with a majority, but it doesn't show one.

Most common error: answering yes because 52 is more than 50. The point estimate is above 50, but the margin of error means values at or below 50 can't be ruled out.

Self-Check

When you're done, check that:

  • Each margin of error is written in percentage points.
  • Your answer to Problem 3 uses the square root rule, not doubling.
  • You found the bias in Problem 4 by asking how the voters were chosen.
  • You can say in one sentence why "52 percent, plus or minus 3" doesn't show a majority.

Knowledge Check 10 min Done

From mortality tables to inference: learning about a population from a sample

This ungraded knowledge check assesses your understanding of how statistics uses a sample to learn about a population. You'll be asked about the history of statistics from Graunt to Fisher, samples and bias, margins of error, and the central limit theorem.

Note: Use this to test yourself, review the feedback on any questions you miss, and retry until you feel confident before moving forward.

5 questions · ungraded · retry as often as you like