Skip to content
Statculator

Statistical TestsStudy Design & Sample Size

Type I vs Type II Errors (Type 1 vs Type 2): Definitions and Examples

Type 1 vs Type 2 errors (Type I and II): false positives vs false negatives, α, β and power, 10 examples, a worked power calculation and how to reduce each.

By Hira Noor, BS Mathematics · Updated · 14 min read

Summary

Type 1 and Type 2 errors (Type I and Type II errors) are the two ways a hypothesis test can reach the wrong conclusion. A Type I error is a false positive: rejecting a null hypothesis that is true. A Type II error is a false negative: failing to reject a null hypothesis that is false. The probability of a Type I error is the significance level α, the probability of a Type II error is β, and power = 1 − β is the chance of detecting a real effect. For a fixed sample size, lowering one error rate raises the other. Example: convicting an innocent defendant is a Type I error; acquitting a guilty one is a Type II error.

Key takeaways

  • Type I error: H₀ is true but you reject it (false positive). Its probability is α, which you choose before the test.
  • Type II error: H₀ is false but you fail to reject it (false negative). Its probability is β, and power = 1 − β.
  • β is not a single number: it depends on α, the sample size, the variability and how large the true effect is.
  • With n fixed, a smaller α means a larger β. Only more data, less noise or a larger effect lowers both.
  • Which error is worse depends on what each mistake costs, so set α and the target power case by case.
  • Memory aid: a Type I error is a false alarm (crying wolf when there is no wolf); a Type II error is a missed alarm (no one comes when the wolf is real).
On this page
  1. What are Type I and Type II errors?
  2. α, β and power
  3. Type 1 and Type 2 error examples
  4. How to remember Type 1 and Type 2 errors
  5. The trade-off between Type I and Type II errors
  6. Worked example: calculating β and power
  7. How to reduce each error
  8. Which error is worse?
  9. Type I and Type II errors in A/B testing
  10. Describing Type 1 and Type 2 errors in AP Statistics
  11. Type III and Type IV errors
  12. Common misconceptions
  13. Frequently asked questions

What are Type I and Type II errors?

A hypothesis test ends in one of two decisions: reject the null hypothesis H₀, or fail to reject it. H₀ itself is either true or false, and you never observe which. That gives four possible outcomes, two correct and two wrong. Whether you write Type I vs Type II errors, Type 1 vs Type 2 errors or Type one and Type two errors, the terms mean the same thing in statistics.

The four outcomes of a hypothesis test
DecisionH₀ is actually trueH₀ is actually false
Reject H₀Type I error (false positive), probability αCorrect decision (true positive), probability 1 − β = power
Fail to reject H₀Correct decision (true negative), probability 1 − αType II error (false negative), probability β

A Type I error claims an effect that does not exist: the test comes out significant although H₀ is true. A Type II error misses an effect that does exist: the test comes out non-significant although H₀ is false. The numbering comes from Neyman and Pearson, who called them errors of the first and second kind. It does not rank them by seriousness.

Each probability is conditional on a different state of the world: α is calculated assuming H₀ is true, and β assuming H₀ is false. They therefore do not add up to 1. The pairs that do add up to 1 share a column of the table: α with 1 − α, and β with power.

The same table appears outside hypothesis testing under other names. In diagnostic testing and machine learning it is the confusion matrix: the false positive rate (1 − specificity) plays the role of α, and the false negative rate (1 − sensitivity) plays the role of β. A screening test with 95% specificity and 80% sensitivity therefore behaves like a test with α = 0.05 and power 0.80.

Type I vs Type II error at a glance
AspectType I errorType II error
Also calledFalse positive, error of the first kind, α errorFalse negative, error of the second kind, β error
What really holdsH₀ is trueH₀ is false
Decision madeReject H₀Fail to reject H₀
In plain wordsFinding an effect that is not thereMissing an effect that is there
Probabilityα, the significance levelβ; power = 1 − β
Who sets itYou choose α before the testFollows from α, n, variability and the true effect
Main way to reduce itSmaller α, multiple-comparison correctionsLarger sample, less noise, larger effect
Courtroom exampleAn innocent defendant is convictedA guilty defendant is acquitted

α, β and power

  • α, the significance level, is the probability of a Type I error: P(reject H₀ | H₀ true). You choose it before collecting data. The most common value is 0.05, with 0.01 and 0.10 also in use.
  • β is the probability of a Type II error: P(fail to reject H₀ | H₀ false). You do not set it directly. It follows from α, the sample size, the variability of the data and how far the truth lies from H₀.
  • Power = 1 − β is the probability that the test detects an effect of a given size. Studies are commonly planned for a power of 0.80 or 0.90, which means accepting β = 0.20 or 0.10.

The link to p-values is direct. You reject H₀ when p ≤ α, so when H₀ is true a p-value at or below 0.05 still turns up in 5% of studies, and each of those is a Type I error. The guide What Is a P-Value? Meaning, Calculation and Common Mistakes covers that decision rule in detail.

Type I and Type II errors on two overlapping normal curves of the sample mean: under H₀ (μ = 100) the area beyond the cut-off 104.93 is α = 0.05; under H₁ (μ = 106) the area below the cut-off is β = 0.36 and the area above it is power = 0.64.
One cut-off, two error rates. The dark green tail is α, the Type I error rate if H₀ is true. The red area is β, the Type II error rate if the true mean is 106. Moving the cut-off right shrinks α and grows β.

Type 1 and Type 2 error examples

In any setting, start by writing down H₀. The Type I error is then “acting as if H₀ were false when it is true”, and the Type II error is the reverse.

Type I and Type II errors in three settings
SettingNull hypothesis H₀Type I error (false positive)Type II error (false negative)
Medical screening testThe patient does not have the diseaseA healthy patient tests positive and is sent for further testsA patient with the disease tests negative and goes untreated
Criminal trialThe defendant is innocentAn innocent defendant is convictedA guilty defendant is acquitted
A/B test of a new checkout pageThe new page converts no better than the old oneThe team ships a page that is no better, believing it isThe team discards a page that really is better
Clinical drug trialThe drug works no better than placeboAn ineffective drug is approvedAn effective drug is abandoned
Email spam filterThe email is legitimateA genuine email lands in the spam folderA spam email reaches the inbox
Smoke alarmThere is no fireThe alarm sounds while someone is cookingThe alarm stays silent during a real fire
Factory quality controlThe batch meets specificationA good batch is scrappedA defective batch is shipped to customers
Airport security scannerThe bag holds no prohibited itemA harmless bag is pulled for a manual searchA prohibited item passes through
Bank fraud detectionThe card transaction is genuineA genuine purchase is declinedA fraudulent purchase goes through
Home pregnancy testThe person is not pregnantThe test reads positive for someone who is not pregnantThe test reads negative for someone who is pregnant

The courtroom shows the asymmetry built into testing. “Innocent until proven guilty” makes innocence the null hypothesis and demands strong evidence before rejecting it, which keeps Type I errors rare at the price of more Type II errors. An acquittal means “not proven guilty”, just as a non-significant result means “fail to reject H₀”, not “H₀ is true”.

The labels depend entirely on how H₀ is stated. In the smoke-alarm row, H₀ is “there is no fire”, so a false alarm is the Type I error. If you reversed the hypotheses, the names of the two errors would swap, which is why the null hypothesis always comes first.

How to remember Type 1 and Type 2 errors

  • The boy who cried wolf. Take H₀ as “there is no wolf”. When the villagers first run out for a wolf that is not there, they make a Type I error. When they later ignore the boy and a real wolf attacks, they make a Type II error.
  • False alarm, then missed alarm. Type I comes first in the numbering and in the story: an alarm with nothing behind it. Type II is the alarm that should have gone off and did not.
  • Link the error to the decision. A Type I error can only happen when you reject H₀; a Type II error can only happen when you fail to reject it. If your test was significant, the only error you can have made is Type I.
  • Tie each Greek letter to its error. α (alpha, the first letter) is the Type I error rate; β (beta, the second letter) is the Type II error rate.

The trade-off between Type I and Type II errors

For a fixed sample size, α and β move in opposite directions. Lowering α pushes the critical value further from what H₀ predicts, so real effects also find it harder to clear, and β rises.

Take a right-tailed z-test of H₀: μ = 100 with σ = 15 and n = 25, when the true mean is 106. Changing only α gives:

Stricter α, larger β (n = 25, σ = 15, true mean 106)
αReject H₀ when x̄ exceedsβ (Type II error)Power
0.10103.840.23620.7638
0.05104.930.36120.6388
0.01106.980.62790.3721

Moving from α = 0.05 to α = 0.01 makes false positives five times rarer, but the chance of missing this effect climbs from 36% to 63%. The only ways to lower both error rates together are to collect more data, to reduce the noise in the measurements, or to study a larger effect.

Worked example: calculating β and power

Power of a right-tailed z-test

Problem. A training program is claimed to raise test scores above the population mean of 100, where σ = 15. You plan to test H₀: μ = 100 against H₁: μ > 100 at α = 0.05 with n = 25 participants. If the program truly raises the mean to 106, what are β and the power?

  1. 01Standard error. SE = σ / √n = 15 / √25 = 3.
  2. 02Rejection rule. The right-tailed critical value at α = 0.05 is z = 1.6449, so H₀ is rejected when x̄ > 100 + 1.6449 × 3 = 104.9346.
  3. 03Switch to the true distribution. If μ = 106, the sample mean is normal with mean 106 and standard error 3. The cut-off lies (104.9346 − 106) / 3 = −0.3551 standard errors from the true mean.
  4. 04Power. P(x̄ > 104.9346 | μ = 106) = P(Z > −0.3551) = 0.6388.
  5. 05Type II error. β = 1 − 0.6388 = 0.3612.

Answer. The test has a 5% chance of a Type I error if the program does nothing, and a 36% chance of a Type II error if it raises the mean by 6 points. Power is about 64%, below the usual 80% target.

The same steps give a general formula for the Type II error of a right-tailed z-test, where Φ is the standard normal cumulative distribution function and z₁₋α is the critical value:

Type II error rate β (right-tailed z-test)
β = Φ( z₁₋α − (μ₁ − μ₀) / (σ / √n) ), power = 1 − β

With z₀.₉₅ = 1.6449, μ₁ − μ₀ = 6 and σ / √n = 3, β = Φ(1.6449 − 2) = Φ(−0.3551) = 0.3612, as above. Sample size and effect size change the picture quickly. Keeping α = 0.05 and σ = 15:

Power of the same test for other sample sizes and true means
Sample sizeTrue mean 103 (d = 0.2)True mean 106 (d = 0.4)True mean 109 (d = 0.6)
n = 250.25950.63880.9123
n = 500.40880.88170.9953
n = 1000.63880.9907> 0.9999

Here d is the effect in standard-deviation units, (μ − μ₀) / σ. Reaching 80% power for the 6-point effect takes n = 39, and 90% power takes n = 54. The same logic drives every power analysis. For a two-sided, two-sample t-test at α = 0.05, detecting a medium effect of d = 0.5 with 80% power needs 64 participants per group (63 by the normal-approximation formula), while 30 per group gives only 48% power.

Try it: Sample Size CalculatorChoose α and the power you want, enter the difference you need to detect, and get the sample size for comparing two means or two proportions.

How to reduce each error

The two errors respond to different levers.

Reducing Type I errors

  • Choose a smaller α, such as 0.01, when a false positive is costly.
  • Correct for multiple comparisons. Twenty independent tests at α = 0.05 have a 64% chance of at least one false positive when every null hypothesis is true (1 − 0.95²⁰ = 0.6415). A Bonferroni-type correction, or one overall test such as ANOVA in place of many t-tests, keeps the overall rate at α.
  • Fix the analysis before seeing the data. Choosing the outcome, the tail or the stopping point after looking at the results pushes the real false-positive rate above the nominal α.

Reducing Type II errors

  • Increase the sample size. The standard error shrinks in proportion to 1/√n, which is the most direct way to raise power. Plan it with the Sample Size Calculator.
  • Reduce variability. More precise measurement, a more homogeneous sample or a paired design (see the Paired t-Test Calculator) lowers the noise the effect has to stand out from.
  • Study a larger effect where you can, for example a stronger dose or a longer exposure. The Effect Size Calculator converts an expected difference into a standardized effect such as Cohen’s d for planning.
  • Use a one-tailed test only when it is justified in advance. It has more power in the predicted direction and none in the other.
  • Raise α, accepting more false positives, when a missed effect is the costlier mistake.

Which error is worse?

Neither is worse in general; it depends on what each mistake costs. The convention of α = 0.05 with 80% power sets β = 0.20, four times α, which treats a false positive as the more serious error. That ratio does not suit every problem.

  • Type I is usually worse when acting on a false claim is expensive or hard to reverse: approving an ineffective drug, convicting an innocent person, or publishing a finding that others will build on. Use a small α.
  • Type II is usually worse when missing a real effect is the bigger loss: a screening test that misses a treatable disease, or a safety check that fails to notice a defect. Accept a larger α, or pay for a larger sample, to gain power.
  • In low-stakes, reversible decisions, such as an A/B test of a button colour, both errors are cheap and a looser α such as 0.10 can be reasonable.

Type I and Type II errors in A/B testing

An A/B test is a hypothesis test with H₀ “the variant performs no better than the control”. A Type I error ships a change that does nothing, and a Type II error throws away a change that works. Two habits make each error far more common than the nominal rates suggest.

  • Peeking inflates Type I errors. Checking the results repeatedly and stopping as soon as p < 0.05 is not a test at α = 0.05. With 5 equally spaced looks the overall false-positive rate is about 14%, and with 10 looks about 19%. Fix the sample size in advance, or use a sequential method designed for repeated looks.
  • Small tests produce Type II errors. Detecting a lift in conversion rate from 10% to 12% at α = 0.05 (two-sided) with 80% power takes about 3,841 visitors per variant. With 1,000 per variant the power is only about 30%, so the test misses a real 2-point lift 70% of the time. Work out the size first with the Sample Size Calculator.
  • Low-power wins are often exaggerated. When an underpowered test does reach significance, the observed lift tends to overstate the true one, a Type M error in the terms used below.

Describing Type 1 and Type 2 errors in AP Statistics

AP Statistics questions usually ask you to describe an error “in context” and to give a consequence. A definition alone (“rejecting a true null hypothesis”) does not earn credit; the answer has to name the parameter, the claim and the decision in the words of the problem.

  1. Write H₀ in context and decide whether the error you are asked about rejects it (Type I) or fails to reject it (Type II).
  2. State what is actually true and what you concluded. Type I: “H₀ is true, but we concluded H₁.” Type II: “H₁ is true, but we did not find convincing evidence for it.”
  3. Add a consequence that follows from acting on the wrong conclusion.

AP-style example: a battery claim

Problem. A manufacturer claims its batteries last 40 hours on average. A consumer group tests H₀: μ = 40 against Hₐ: μ < 40, where μ is the mean life of all the manufacturer’s batteries. Describe a Type I and a Type II error in context, with a consequence of each.

  1. 01Type I error. The mean life of all the batteries really is 40 hours, but the group finds convincing evidence that it is less than 40. Consequence: the group publishes a false complaint and damages an honest manufacturer.
  2. 02Type II error. The mean life of all the batteries really is less than 40 hours, but the group does not find convincing evidence of it. Consequence: consumers keep buying batteries that fall short of the advertised life.

Answer. Each answer names the parameter (mean life of all batteries), the truth, the conclusion and a consequence. Writing “fail to find convincing evidence” rather than “accept H₀” keeps the wording correct.

Type III and Type IV errors

Only Type I and Type II errors are part of the Neyman–Pearson theory of testing. Later authors proposed further “types”, and their definitions are not standardized, so check which meaning a source intends.

Informal error types beyond Type I and Type II
TermUsual meaningExample
Type III errorCorrectly rejecting H₀ for the wrong reason (Mosteller, 1948), concluding the wrong direction after a two-sided test (Kaiser, 1966), or giving the right answer to the wrong problem (Kimball, 1957)A two-sided test is significant and the report says the drug raised blood pressure when it lowered it
Type IV errorMisinterpreting a correctly rejected hypothesis, such as choosing the wrong follow-up test (Marascuilo and Levin, 1970)An ANOVA rejects H₀ but the wrong pair of groups is named as different
Type S and Type M errorsGetting the sign (S) or the magnitude (M) of an effect wrong, an idea from Gelman and Carlin (2014) for low-power studiesA small study reports a large effect that is mostly noise

Common misconceptions

  • “α is the probability that a significant result is a false positive.” α is the probability of rejecting H₀ when H₀ is true. The share of significant results that are false positives also depends on how often the tested hypotheses are true and on the power of the studies.
  • “α + β = 1.” They are conditional on different states of the world. In the worked example α = 0.05 and β = 0.3612.
  • “A non-significant result means there is no effect.” It may be a Type II error. With 30 per group and a medium effect, a t-test misses the effect more often than it finds it.
  • “A very small α makes a test safer.” It makes Type I errors rarer and Type II errors more common, unless the sample grows to compensate.
  • “The data show which error was made.” They cannot. A single study never reveals whether H₀ is true; the error rates describe how the procedure performs over many studies.
  • “Power can be worked out afterwards from the observed effect.” Power computed from the observed effect only restates the p-value. Power is a planning tool, calculated for an effect size chosen before the study.

Frequently asked questions

What is an example of a Type 1 and a Type 2 error?

In a criminal trial where H₀ is “the defendant is innocent”, convicting an innocent defendant is a Type I error and acquitting a guilty one is a Type II error. In medical screening, telling a healthy patient they have the disease is a Type I error, and missing the disease in a patient who has it is a Type II error.

What is the difference between a Type I and a Type II error?

A Type I error is rejecting a null hypothesis that is true, a false positive. A Type II error is failing to reject a null hypothesis that is false, a false negative. The first claims an effect that is not there, and the second misses one that is.

What is the probability of a Type I error?

The probability of a Type I error is the significance level α, which the researcher chooses before the test. With α = 0.05, a true null hypothesis is rejected in 5% of studies in the long run. The figure applies only when the null hypothesis is true.

How are β and power related?

Power equals 1 − β. β is the probability of missing a real effect of a stated size, and power is the probability of detecting it, so a test with β = 0.20 has 80% power. Both depend on the effect size, the sample size, the variability and α.

Does increasing the sample size reduce both types of error?

A larger sample reduces the Type II error rate β for any fixed α, because the standard error shrinks and real effects stand out more clearly. The Type I error rate stays at whatever α you choose. A larger sample therefore lets you lower α without losing power, which is the only way to reduce both together.

Is a Type I or a Type II error worse?

It depends on the consequences. A Type I error is worse when acting on a false finding is costly, as in approving an ineffective drug. A Type II error is worse when missing a real effect is costly, as in a screening test that fails to detect a disease. Set α and the target power to reflect those costs.

How can I remember which error is which?

Use the boy who cried wolf. With H₀ “there is no wolf”, the villagers who run out for a wolf that is not there make a Type I error (a false alarm), and the villagers who ignore the boy when the wolf is real make a Type II error (a missed alarm). α, the first Greek letter, goes with the first error.

What are Type 1, 2 and 3 errors?

A Type 1 error rejects a true null hypothesis, and a Type 2 error fails to reject a false one. A Type 3 error is an informal term with several meanings: correctly rejecting the null hypothesis for the wrong reason, concluding the wrong direction after a two-sided test, or giving the right answer to the wrong problem. Only Types 1 and 2 are part of standard hypothesis-testing theory.

What is a Type 4 error?

A Type 4 error, proposed by Marascuilo and Levin in 1970, is the incorrect interpretation of a correctly rejected hypothesis, for example running the wrong follow-up comparison after a significant ANOVA. It is not a standard term, and textbooks rarely use it.

Further reading

Cite this page

Statculator. (2026, November 14). Type I vs Type II Errors (Type 1 vs Type 2): Definitions and Examples. Retrieved November 14, 2026, from https://statculator.com/learn/type-i-vs-type-ii-errors/