Introduction
p-values appear in news reports of studies, drug approvals, and scientific papers, usually with little explanation. This walkthrough computes and reads one in a setting where you can see every piece: a coin that may or may not be fair. You'll state the assumption a p-value rests on, see where the number comes from, read it in plain language, and then check that reading against the American Statistical Association's 2016 principles.
The Starting Point
You toss a coin 100 times and it lands heads 60 times. A fair coin should land heads about half the time, so you'd expect about 50. Sixty is more than that. The question is whether 60 is far enough from 50 to make you doubt the coin, or whether a fair coin could produce it by chance.
A few terms first. A p-value is the probability, calculated under the assumption of no real effect, of getting data at least as extreme as the data actually observed. Here, "no real effect" means the coin is fair. A result is called statistically significant when its p-value falls below a fixed cutoff, most often 0.05.
Walking Through the Test
Step 1: State the "no effect" assumption
Every p-value starts from a specific assumption, often called the null hypothesis. Here it is: the coin is fair, so each toss has probability 1/2 of heads, and tosses don't affect each other.
You're not claiming the coin is fair. You're adopting the assumption provisionally, to see how well the data fit it. The ASA statement describes a p-value as a measure of how incompatible the data are with a specified statistical model (Wasserstein and Lazar 2016). The fair-coin assumption is that model.
Step 2: Ask how often a fair coin gives a result at least this far from 50
Under the fair-coin assumption, the number of heads in 100 tosses varies around 50. Its typical spread, the standard deviation, is 5 heads. (For coin tosses, that's the square root of 100 \times \frac{1}{2} \times \frac{1}{2} = 25, and \sqrt{25} = 5.) So 60 heads is two spreads above 50.
"At least as extreme" needs a decision. A coin biased toward tails would be as suspicious as one biased toward heads, so a result 10 or more away from 50 in either direction counts: 60 or more heads, or 40 or fewer. This is a two-sided test.
The probability of each possible number of heads for a fair coin can be computed exactly from counting rules. Adding up the probabilities for 60 heads or more gives about 0.028. The chance of 40 heads or fewer is the same, by symmetry. So
p \approx 0.028 + 0.028 \approx 0.057
A fair coin tossed 100 times lands at least 10 away from 50 heads about 5.7 percent of the time. A rough check agrees: in a bell-shaped distribution, about 5 percent of results fall more than two spreads from the center.
Step 3: Read the p-value
A p-value of about 0.057 says: if the coin were fair, a result as lopsided as yours, or more so, would turn up in about 1 of every 17 or 18 experiments like this one. That's unusual but not rare. If many people each tossed a fair coin 100 times, several in a hundred would see a result like yours.
So the data are somewhat surprising under the fair-coin assumption. They give you a modest reason to wonder about the coin, not a strong one.
Step 4: Apply the ASA principles
Two of the ASA's principles bear directly on this result (Wasserstein and Lazar 2016).
First, a p-value doesn't measure the probability that the hypothesis being studied is true. The 0.057 was calculated by assuming the coin is fair. It describes the data under that assumption. It can't then turn around and tell you how likely the assumption is.
Second, conclusions shouldn't rest only on whether a p-value passes a specific threshold. Under the 0.05 convention, 0.057 is "not significant." But compare what one more or one fewer head would have done:
| Heads in 100 tosses | Two-sided p-value | Label under the 0.05 convention |
|---|
| 59 | about 0.089 | Not significant |
| 60 | about 0.057 | Not significant |
| 61 | about 0.035 | Significant |
A single toss moves the result across the line. The evidence from 60 heads and from 61 heads is nearly the same, so treating them as opposite verdicts gives the cutoff more weight than it can bear.
Step 5: Decide what more evidence would help
The p-value can't settle the question, but it can tell you what would. Two kinds of evidence would help.
More tosses. If the coin really lands heads 60 percent of the time, more tosses would make that clearer. With 200 tosses and 120 heads, the same 60 percent, the two-sided p-value drops to about 0.006. With 1,000 tosses and 600 heads, it's far smaller than one in a million.
A reason to suspect the coin. Is it bent? Was it supplied by someone with a stake in the outcome? Was it tossed or spun? A p-value only describes data under an assumption. Knowledge about how the data were produced can make a modest p-value more or less worrying.
Key Considerations
The p-value depends on the assumption. Change the "no effect" model and the p-value changes. If tosses weren't independent, for example, the fair-coin calculation wouldn't apply.
One-sided or two-sided matters. If you had decided in advance that only a bias toward heads mattered, you would count only 60 or more heads, giving a one-sided p-value of about 0.028. The choice has to be made before seeing the data. Choosing after, to get a smaller number, makes the p-value misleading.
The common mistake: reading "p = 0.057" as "a 5.7 percent chance the coin is fair." This reverses the calculation. The p-value is the chance of data this extreme if the coin is fair, not the chance the coin is fair given these data. The ASA statement rules out this reading (Wasserstein and Lazar 2016). Working out the probability that the coin is fair would need more information, such as how common biased coins are where this one came from.
Summary
A coin landing heads 60 times in 100 gives a two-sided p-value of about 0.057. Here is the model result:
p-value: about 0.057, two-sided, under the assumption that the coin is fair.
Plain-language reading: If the coin were fair, a result at least this far from 50 heads would happen about 5.7 percent of the time. The result is unusual, not rare, and gives modest reason to doubt the coin.
What it doesn't tell you: (1) the probability that the coin is fair; (2) how biased the coin is, if it is biased at all; (3) whether the coin is fair or unfair, decided by which side of 0.05 the number falls on.
- The calculation adds the fair-coin probabilities of 60 or more heads (about 0.028) and 40 or fewer (about 0.028).
- The reading describes the data under the assumption, never the assumption itself.
- The three limits follow the ASA principles that a p-value isn't the probability a hypothesis is true, doesn't measure the size of an effect, and shouldn't decide a conclusion by passing a threshold.
References
- Wasserstein, Ronald L., and Nicole A. Lazar. 2016. "The ASA's Statement on p-Values: Context, Process, and Purpose." The American Statistician 70 (2): 129–133. doi:10.1080/00031305.2016.1154108.