Skip to content
Statculator

Statistical Tests

How to Choose the Right Statistical Test

How to choose a statistical test in three decisions: goal, outcome type and design, plus a decision table that maps 23 situations to the right test.

By Hira Noor, BS Mathematics · Updated · 13 min read

Summary

To choose a statistical test, answer three questions in order. What is your goal: comparing groups, measuring a relationship, checking fit to an expected distribution, or predicting an outcome? What type is the outcome variable: continuous, categorical or ordinal? And how is the study designed: how many groups, and are the measurements independent or paired? The answers point to one test, such as a t-test, ANOVA, chi-square test, correlation or regression. A final assumption check decides between that test and its rank-based (nonparametric) alternative.

Key takeaways

  • Start from the research question, not the data file: compare, relate, fit or predict.
  • The type of the outcome variable (continuous, categorical or ordinal) narrows the choice more than anything else.
  • Paired designs need paired tests; independent groups need independent-samples tests.
  • Use one ANOVA, not a series of t-tests, for three or more groups.
  • Means call for t-tests or ANOVA; counts in categories call for chi-square or proportion tests.
  • Switch to a rank-based (nonparametric) test when the outcome is ordinal or small samples are clearly non-normal.
On this page
  1. What a statistical test does
  2. How to choose a statistical test in three decisions
  3. Decision 1: What is your goal?
  4. Decision 2: What type is your outcome variable?
  5. Decision 3: How many groups, and are they paired?
  6. Which statistical test to use: decision table
  7. t-test, ANOVA or chi-square?
  8. Parametric or nonparametric?
  9. Check the assumptions before you run the test
  10. Worked scenarios
  11. After the test: effect size and sample size
  12. Running the test in R, Python or SPSS
  13. Common mistakes when choosing a test
  14. Frequently asked questions

What a statistical test does

A statistical test measures how compatible your data are with a specific null hypothesis, such as “the two group means are equal” or “the two variables are unrelated.” It turns the data into a test statistic (t, F, χ², z or r), compares that statistic with the distribution it would follow if the null hypothesis were true, and reports a p-value: the probability of a result at least as extreme as yours under that null hypothesis.

Each test is built for one kind of question, one kind of data and one kind of design. Choosing the wrong test rarely produces an error message. It produces a p-value that answers a different question from the one you asked, which is why the choice matters more than the arithmetic.

The main types of statistical tests follow the four goals below: comparison tests (z-tests, t-tests, ANOVA), correlation tests (Pearson, Spearman), tests on categorical counts (chi-square, Fisher’s exact) and regression models. Each comes in a parametric form and, for most designs, a rank-based nonparametric form.

How to choose a statistical test in three decisions

Work through three decisions in order, then run one assumption check:

  1. 1. GoalCompare groups, measure a relationship, test fit or predict?
  2. 2. Outcome typeContinuous, categorical or ordinal?
  3. 3. DesignHow many groups? Independent or paired?
  4. Assumption checkParametric test, or its rank-based alternative?
The decision path used throughout this guide. Each step narrows the list of candidate tests; the last step picks between a test and its nonparametric version.
  1. Goal. Are you comparing groups, measuring a relationship between variables, checking whether counts fit an expected distribution, or predicting an outcome?
  2. Outcome type. Is the variable you measured continuous (time, weight, score), categorical (yes/no, brand chosen) or ordinal (a ranking or a 1–5 rating)?
  3. Design. How many groups or samples are there, and are they independent (different subjects) or paired (the same subjects measured twice, or matched pairs)?

The final check decides between a standard (parametric) test and a rank-based (nonparametric) alternative: are the test’s assumptions about normality and equal spread reasonable for your data? The assumptions section below shows how to check.

Statistical test chooser: answer the questions to get a suggestion

Suggested test

Welch’s t-test

Safe default; the pooled two-sample t-test is fine when the standard deviations are similar.

Run it: Welch’s t-Test Calculator

The chooser covers the common cases in this guide. Read the sections below to see why each answer leads where it does, and check the assumptions before you report a result.

Decision 1: What is your goal?

Most questions in applied statistics fall under four goals. Write your question as one sentence first; the verb usually tells you which goal it is.

Four goals and the test families that serve them
GoalTypical questionTest family
Compare groupsDo customers who saw design B spend more than those who saw design A?z-tests, t-tests, ANOVA, proportion tests
Measure a relationshipIs study time associated with exam score?Pearson or Spearman correlation; chi-square test of independence for categories
Test goodness of fitAre the six faces of this die equally likely?Chi-square goodness-of-fit test
Predict an outcomeHow much does price rise for each extra square metre of floor space?Simple, multiple or logistic regression

Comparison and relationship questions overlap. “Does smoking status differ by age group?” compares proportions across groups and also asks whether two categorical variables are associated. The chi-square test of independence answers both forms of the question.

Decision 2: What type is your outcome variable?

The outcome (dependent) variable is the thing you measured and want to explain. Its type decides which summary the test compares: means for continuous (quantitative) data, proportions or counts for categorical data, and ranks for ordinal data.

Variable types and the tests that fit them
Variable typeExamplesWhat the test comparesTypical tests
ContinuousWeight, reaction time, revenue, test scoreMeans (or variances)t-tests, z-tests, ANOVA, F-test, Pearson correlation, linear regression
BinaryConverted or not, pass or failProportionsProportion z-tests, chi-square, Fisher’s exact, McNemar’s test, logistic regression
Nominal (3+ categories)Blood type, preferred brandCounts per categoryChi-square goodness of fit, chi-square test of independence
OrdinalSatisfaction on a 1–5 scale, finishing positionRanksMann–Whitney U, Wilcoxon signed-rank, Kruskal–Wallis, Spearman correlation

A fifth type, time-to-event data such as time until relapse or until a machine fails, needs survival methods: the log-rank test to compare groups and Cox regression to model the outcome. These handle subjects who have not yet had the event when the study ends, which ordinary t-tests cannot.

Decision 3: How many groups, and are they paired?

For comparison questions, the design picks the specific test within a family.

  • One sample against a fixed value. You compare one sample’s mean or proportion with a target, a historical figure or a claim. Example: is the mean fill volume different from 500 ml? See the one-sample t-test guide.
  • Two independent groups. Each subject belongs to exactly one group, and the groups do not influence each other. Example: a treatment group and a separate control group. See the independent-samples t-test guide.
  • Two paired measurements. Each value in one set is linked to one value in the other: before and after on the same person, left eye and right eye, or matched case–control pairs. The analysis works on the within-pair differences. See the paired t-test guide.
  • Three or more groups. Run one overall test (ANOVA for means, chi-square for proportions) and follow up with post-hoc comparisons, rather than testing every pair separately.
  • Two grouping factors at once. When groups are defined by two factors, such as fertilizer and watering level, a two-way ANOVA tests each factor and their interaction. See one-way vs two-way ANOVA.

Which statistical test to use: decision table

Find the row that matches your goal, outcome and design. Where Statculator has a calculator for the test, the last column links to it; each calculator shows the full working.

Choosing a statistical test by goal, outcome and design
Goal and designOutcomeTestCalculator
One mean vs a target value, σ unknownContinuousOne-sample t-testOne-Sample t-Test Calculator
One mean vs a target value, σ knownContinuousOne-sample z-testOne-Sample Z-Test Calculator
Two independent means, similar variancesContinuousTwo-sample (pooled) t-testTwo-Sample t-Test Calculator
Two independent means, unequal variancesContinuousWelch’s t-testWelch’s t-Test Calculator
Two independent groups, skewed or ranked dataOrdinal or continuousMann–Whitney U testSoftware (see below)
Two paired measurementsContinuousPaired t-testPaired t-Test Calculator
Two paired measurements, skewed differencesOrdinal or continuousWilcoxon signed-rank testSoftware (see below)
Three or more independent meansContinuousOne-way ANOVAANOVA Calculator
Three or more independent groups, skewed or ranked dataOrdinal or continuousKruskal–Wallis H testSoftware (see below)
Two grouping factors and their interactionContinuousTwo-way ANOVATwo-way ANOVA guide
Same subjects measured three or more timesContinuousRepeated-measures ANOVA (Friedman test if ranked)Software (see below)
Two variancesContinuousF-test for equal variancesF-Test Calculator
One proportion vs a claimed valueBinaryOne-proportion z-testOne Proportion Z-Test Calculator
Two independent proportionsBinaryTwo-proportion z-testTwo Proportion Z-Test Calculator
Two paired proportions (before and after on the same people)BinaryMcNemar’s testSoftware (see below)
Association between two categorical variablesCategoricalChi-square test of independenceChi-Square Test of Independence Calculator
2×2 table with small expected countsBinaryFisher’s exact testFisher’s Exact Test Calculator
Observed counts vs an expected distributionCategoricalChi-square goodness of fitChi-Square Goodness of Fit Calculator
Linear relationship between two variablesContinuousPearson correlation testPearson’s Correlation Test Calculator
Monotonic relationship, ranked data or outliersOrdinal or continuousSpearman rank correlationSpearman Correlation Calculator
Predict an outcome from one predictorContinuousSimple linear regressionLinear Regression Calculator
Predict an outcome from several predictorsContinuousMultiple linear regressionMultiple Linear Regression Calculator
Predict a yes/no outcomeBinaryLogistic regressionLogistic Regression Calculator

Two rules sit behind several rows. Use a z-test for a mean only when the population standard deviation is genuinely known; otherwise use t. And a chi-square test needs adequate expected counts, commonly at least 5 per cell; when a 2×2 table falls short, switch to Fisher’s exact test.

Try it: T-Test CalculatorNot sure which t-test fits your design? This calculator runs one-sample, pooled, Welch and paired t-tests from the same screen.

t-test, ANOVA or chi-square?

These three tests cover most introductory comparisons, and the choice between them comes down to two facts: what the outcome is, and how many groups you have. A t-test and ANOVA compare means of a continuous outcome. A chi-square test compares counts of a categorical outcome.

t-test vs ANOVA vs chi-square test
Featuret-testOne-way ANOVAChi-square test of independence
OutcomeContinuousContinuousCategorical
Groups comparedOne or twoThree or moreTwo or more categories of each variable
What it comparesMeansMeansObserved vs expected counts
Test statistictFχ²
Example questionDo two diets give different mean weight loss?Do four fertilizers give different mean yields?Is preferred brand related to age group?

A quick way to tell them apart is to look at one row of your data. If the outcome cell holds a number you could average, such as 72.4 kg, use a t-test or ANOVA. If it holds a label, such as “Brand B,” and your analysis counts how often each label appears, use chi-square. Converting a continuous outcome into categories just to run a chi-square test throws away information and usually loses power.

Parametric or nonparametric?

Parametric tests such as the t-test and ANOVA assume that the data, or more precisely the sampling distribution of the mean, are approximately normal, and some also assume equal variances. Nonparametric tests replace the values with their ranks, so they need weaker assumptions and are less affected by outliers. The cost is some loss of power when the parametric assumptions do hold, and a slightly different hypothesis: rank tests compare whole distributions or typical values rather than means.

Parametric tests and their rank-based alternatives
Parametric testNonparametric alternativeDesign
One-sample t-testWilcoxon signed-rank testOne sample vs a value
Paired t-testWilcoxon signed-rank test on the differencesTwo paired measurements
Two-sample or Welch’s t-testMann–Whitney U (Wilcoxon rank-sum) testTwo independent groups
One-way ANOVAKruskal–Wallis H testThree or more independent groups
Repeated-measures ANOVAFriedman testThree or more related measurements
Pearson correlationSpearman rank correlationTwo variables

Prefer the nonparametric version when the outcome is ordinal, when groups are small (roughly under 15–20 each) and clearly skewed, or when a few extreme values dominate. With moderate or large samples, t-tests and ANOVA hold up well against non-normality because of the central limit theorem, so mild skew alone is not a reason to switch.

Check the assumptions before you run the test

Once the three decisions point to a test, check that its assumptions are reasonable. Do this by looking at the data, not by trying several tests and keeping the one with the smallest p-value. In formal research, name the main test and its fallback in the protocol or statistical analysis plan before the data are analysed.

Common assumptions and how to check them
AssumptionApplies toHow to checkIf it fails
Independent observationsAlmost every testStudy design: one value per subject, no clusteringUse a paired, repeated-measures or multilevel method
Approximate normalityt-tests, ANOVA, Pearson testPlot a histogram or box plot; for small samples, a Shapiro–Wilk testUse the rank-based alternative, or rely on a large sample
Equal variancesPooled t-test, one-way ANOVACompare standard deviations; run Levene’s testUse Welch’s t-test or Welch’s ANOVA
Expected counts of at least 5Chi-square testsCompute expected counts for every cellFisher’s exact test (2×2) or combine sparse categories
Linear relationshipPearson correlation, linear regressionDraw a scatter plotSpearman correlation or a curved model

Worked scenarios

Each scenario applies the three decisions in order.

Blood pressure before and after a program

Problem. A clinic measures systolic blood pressure for 25 patients before and after a 12-week exercise program. Did blood pressure change?

  1. 01Goal: compare two sets of measurements.
  2. 02Outcome: blood pressure in mmHg, a continuous variable.
  3. 03Design: the same 25 patients are measured twice, so the sets are paired.
  4. 04Assumption check: plot the 25 differences (after − before). If they are roughly symmetric with no extreme outliers, a t-test is appropriate.

Answer. Use a paired t-test on the differences (Paired t-Test Calculator). If the differences are strongly skewed, use the Wilcoxon signed-rank test instead.

Landing-page conversion

Problem. An online store shows page A to 2,000 visitors and page B to a different 2,000 visitors, and records whether each visitor bought something. Do the conversion rates differ?

  1. 01Goal: compare two groups.
  2. 02Outcome: bought or did not buy, a binary variable, so the summary is a proportion.
  3. 03Design: different visitors saw each page, so the groups are independent.
  4. 04Sample size: thousands of visitors per page keep expected counts far above 5.

Answer. Use a two-proportion z-test (Two Proportion Z-Test Calculator). A chi-square test on the 2×2 table without continuity correction gives the same two-sided p-value, because χ² = z² for this design.

Four fertilizers

Problem. An agronomist applies four fertilizers to six plots each and records wheat yield in tonnes per hectare. Does yield depend on fertilizer?

  1. 01Goal: compare groups.
  2. 02Outcome: yield, a continuous variable.
  3. 03Design: four independent groups, so a two-group t-test is not enough.
  4. 04Follow-up: if the overall test is significant, compare pairs with a post-hoc method such as Tukey’s HSD.

Answer. Use a one-way ANOVA (ANOVA Calculator) with df = 3 and 20. The one-way ANOVA guide works through the full calculation.

Satisfaction ratings at three branches

Problem. A bank asks 12 customers at each of three branches to rate their visit from 1 (very poor) to 5 (excellent). Most ratings are 4 or 5. Do satisfaction levels differ between branches?

  1. 01Goal: compare groups.
  2. 02Outcome: a single 1–5 rating, an ordinal variable.
  3. 03Design: three independent groups of different customers.
  4. 04Assumption check: with 12 ratings per branch piled up at 4 and 5, the data are ordinal and skewed, so a test on ranks fits better than one on means.

Answer. Use a Kruskal–Wallis H test, the rank-based counterpart of one-way ANOVA, with df = 3 − 1 = 2. If it is significant, follow up with pairwise rank tests and a correction for multiple comparisons.

Sleep and exam scores

Problem. A lecturer records hours of sleep the night before an exam and the exam score for 40 students. Are the two related, and can sleep predict the score?

  1. 01Goal: first a relationship, then a prediction.
  2. 02Variables: both continuous.
  3. 03Plot first: a scatter plot shows whether the pattern is roughly linear and whether a few points drive it.
  4. 04If the pattern is linear, test the correlation, then fit a line to predict score from sleep.

Answer. Use a Pearson correlation test (Pearson’s Correlation Test Calculator) for the association and simple linear regression (Linear Regression Calculator) for prediction. If the pattern is curved but consistently rising, use Spearman’s correlation.

Is the die fair?

Problem. A die is rolled 120 times and the count for each face is recorded. Is every face equally likely?

  1. 01Goal: compare observed counts with an expected distribution.
  2. 02Outcome: the face shown, a categorical variable with six levels.
  3. 03Expected counts: 120 / 6 = 20 per face, comfortably above 5.

Answer. Use a chi-square goodness-of-fit test (Chi-Square Goodness of Fit Calculator) with df = 6 − 1 = 5.

After the test: effect size and sample size

A p-value says whether an effect is distinguishable from zero, not whether it is large. Report an effect size next to every test: Cohen’s d for a difference in means, η² for ANOVA, Cramér’s V for a chi-square table, or r itself for a correlation. The Effect Size Calculator computes the common ones.

The same choices also shape planning. A paired design or a parametric test usually needs fewer subjects than the alternative to detect the same effect, and the significance level and power you choose set the risk of a false positive or a missed effect; see Type I vs Type II errors. Use the Sample Size Calculator to size a study before collecting data.

Running the test in R, Python or SPSS

Once you know the test, every major package runs it in one command or menu. The names below are the standard functions; check each one’s defaults, such as whether a t-test assumes equal variances or whether a continuity correction is applied.

Where to find common tests in statistical software
TestRPython (scipy.stats)SPSS menu
One-sample, Welch or paired t-testt.test()ttest_1samp, ttest_ind, ttest_relAnalyze → Compare Means
One-way ANOVAaov() or oneway.test()f_onewayAnalyze → Compare Means → One-Way ANOVA
Mann–Whitney U, Wilcoxon signed-rankwilcox.test()mannwhitneyu, wilcoxonAnalyze → Nonparametric Tests
Kruskal–Walliskruskal.test()kruskalAnalyze → Nonparametric Tests
Chi-square test of independencechisq.test()chi2_contingencyAnalyze → Descriptive Statistics → Crosstabs
Fisher’s exact testfisher.test()fisher_exactCrosstabs → Exact
Pearson or Spearman correlationcor.test()pearsonr, spearmanrAnalyze → Correlate → Bivariate
McNemar’s testmcnemar.test()statsmodels: mcnemarCrosstabs → Statistics → McNemar
Friedman testfriedman.test()friedmanchisquareAnalyze → Nonparametric Tests

R’s t.test() runs Welch’s test unless you set var.equal = TRUE, and its chisq.test() applies Yates’ continuity correction to 2×2 tables by default. Those defaults explain most small differences between software output and a hand calculation.

Common mistakes when choosing a test

  • Running many t-tests instead of one ANOVA. Three groups give three pairwise comparisons. If they were independent tests at α = 0.05, the chance of at least one false positive would be 1 − 0.95³ ≈ 0.143, nearly three times the intended rate.
  • Ignoring pairing. Before-and-after data analysed with an independent-samples test usually loses power, sometimes dramatically.
  • Choosing the tail after seeing the data. Decide between a one-tailed and a two-tailed test before looking at the results. Switching to one tail because the effect went the “right” way halves the p-value without justification; see What Is a P-Value? Meaning, Calculation and Common Mistakes.
  • Using Pearson’s r for a curved relationship. Pearson’s r measures linear association only, and a strong U-shaped pattern can give r close to 0. Plot the data first.
  • Applying a chi-square test to sparse tables. When expected counts fall below 5, the chi-square approximation becomes unreliable. Fisher’s exact test avoids the problem for 2×2 tables.
  • Running a chi-square test on means or percentages. Chi-square works on raw counts. Feeding it averages or percentages gives a meaningless statistic.
  • Testing a whole population. If you have data on every member of the group you care about, there is no sampling error to test. Describe the data instead; see Descriptive vs Inferential Statistics: Differences and Examples.

Frequently asked questions

What does a statistical test do?

A statistical test measures how compatible your data are with a null hypothesis, such as “the group means are equal.” It converts the data into a test statistic, compares it with the distribution expected under that hypothesis, and reports a p-value. A small p-value means data like yours would be unusual if the null hypothesis were true.

What are the main types of statistical tests?

Most tests fall into four families: comparison tests (z-tests, t-tests, ANOVA and their rank-based alternatives), correlation tests (Pearson and Spearman), tests on categorical counts (chi-square and Fisher’s exact tests) and regression models for prediction. Each family has parametric versions, which assume approximate normality, and nonparametric versions, which work on ranks.

Should I use ANOVA or chi-square?

Use ANOVA when the outcome is continuous and you compare its mean across three or more groups, such as mean yield under four fertilizers. Use a chi-square test when the outcome is categorical and you compare counts, such as how many customers in each age group prefer each brand. The type of outcome variable decides it.

How do I know when to use a t-test or a chi-square test?

Use a t-test when the outcome is a number you can average and you compare the means of one or two groups. Use a chi-square test when the outcome is a category and you compare how often each category occurs. If you would summarize the outcome with a mean, it is a t-test; if you would summarize it with counts or percentages, it is chi-square.

Should I use a t-test or a z-test?

Use a z-test for a mean only when the population standard deviation σ is known from outside the sample, which is rare in practice. When σ is estimated from the sample standard deviation s, use a t-test. For proportions, z-tests are standard because the standard error follows from the hypothesized proportion. With large samples the two give nearly identical results.

When should I use ANOVA instead of a t-test?

Use one-way ANOVA when you compare the means of three or more independent groups. It tests all the groups at once and keeps the false-positive rate at α, whereas running every pairwise t-test inflates it. With exactly two groups, ANOVA and the pooled two-sample t-test give the same p-value, because F = t².

Which test should I use for categorical data?

For counts in categories, use a chi-square goodness-of-fit test to compare one variable with an expected distribution, or a chi-square test of independence to check whether two categorical variables are associated. For a 2×2 table with small expected counts, use Fisher’s exact test. To compare one or two proportions directly, use a one- or two-proportion z-test.

Do I need normally distributed data for a t-test?

Not strictly. The t-test needs the sampling distribution of the mean to be approximately normal, which holds when the population is roughly normal or the sample is reasonably large. With small samples that are clearly skewed or contain outliers, use a rank-based test such as the Mann–Whitney U or Wilcoxon signed-rank test instead.

What is the difference between correlation and regression?

Correlation measures the strength and direction of an association between two variables and treats them symmetrically. Regression models one variable as a function of one or more others and gives an equation you can use for prediction. In simple linear regression, the test of the slope and the Pearson correlation test give the same p-value.

Can I choose a test after looking at my data?

Look at the data to check assumptions such as skewness, outliers or unequal spread, but fix the hypothesis, the tail and the main test from the research question before seeing the results. Switching tests to obtain a smaller p-value makes the reported p-value misleading.

Further reading

Cite this page

Statculator. (2026, November 14). How to Choose the Right Statistical Test. Retrieved November 14, 2026, from https://statculator.com/learn/how-to-choose-a-statistical-test/