Is it real or just chance?
A coin lands heads 60 times in 100 flips. Is it unfair, or was that luck? Hypothesis testing gives a structured way to decide.
Step 1: State the hypotheses
- Null hypothesis H₀: nothing special is happening. (The coin is fair: p = 0.5.)
- Alternative hypothesis Hₐ: something is happening. (p ≠ 0.5, or p > 0.5.)
We assume H₀ is true and ask whether the data contradict it.
Step 2: Choose a significance level
The significance level α is how much evidence you need. Common choice: α = 0.05.
Step 3: Calculate a test statistic
For a proportion:
z = (p̂ − p₀) / √[p₀(1 − p₀)/n]
Coin example: p̂ = 0.60, p₀ = 0.5, n = 100
- SE = √(0.25/100) = 0.05
- z = (0.60 − 0.50)/0.05 = 2.0
Step 4: Find the p-value
The p-value is the probability of results at least this extreme if H₀ were true.
For a two-sided test with z = 2.0: p-value ≈ 2 × 0.0228 ≈ 0.046.
Step 5: Decide and conclude
| p-value vs α | Decision |
|---|---|
| p-value ≤ α | reject H₀: evidence for Hₐ |
| p-value > α | fail to reject H₀: not enough evidence |
0.046 < 0.05 → reject H₀. There’s evidence the coin is not fair (at the 5% level).
Conclude in context: "The data provide evidence that the coin doesn’t land heads 50% of the time."
"Fail to reject" ≠ "accept"
Not finding evidence against H₀ doesn’t prove it true. It just means the data aren’t convincing enough.
Errors
| H₀ true | H₀ false | |
|---|---|---|
| Reject H₀ | Type I error (false alarm) | correct |
| Fail to reject H₀ | correct | Type II error (missed effect) |
The probability of a Type I error is α. Lowering α reduces Type I errors but increases Type II errors. Larger samples reduce both.
Testing a mean
z = (x̄ − μ₀) / (σ/√n)
Example: A company claims batteries last 50 h. A sample of 36 has x̄ = 48.2, σ = 6. Test Hₐ: μ < 50 at α = 0.05.
- z = (48.2 − 50)/(6/6) = −1.8
- p-value (one-sided) ≈ 0.036 < 0.05 → reject H₀. Evidence the mean life is less than 50 h.
Statistical vs practical significance
With huge samples, tiny differences become statistically significant but may not matter in real life. Always ask whether the effect is large enough to care about.
Worked examples
Example 1: p-value = 0.12, α = 0.05. Decision? Fail to reject H₀.
Example 2: H₀: a new drug has no effect. A Type I error means? Concluding it works when it doesn’t.
Example 3: z = 2.5 (two-sided). p-value? ≈ 2 × 0.0062 = 0.012.
Common mistakes
- Saying the p-value is the probability H₀ is true.
- "Accepting" H₀.
- Choosing α after seeing the results.
Try it yourself
- State H₀ and Hₐ: testing whether more than 60% of students like a new schedule.
- p-value 0.03 at α = 0.01. Decision?
- What is a Type II error in a fire alarm analogy?
- A coin lands heads 56 of 100 times. z-value for testing p = 0.5?
Answers: 1) H₀: p = 0.6; Hₐ: p > 0.6 2) fail to reject H₀ 3) The alarm doesn’t sound when there is a fire. 4) z = 1.2
Quick recap
- Hypothesis testing weighs data against a null hypothesis.
- Compare the p-value to the significance level.
- Beware Type I errors and Type II errors.