How to Choose the Right Statistical Test
How to choose a statistical test in three decisions: goal, outcome type and design, plus a decision table that maps 23 situations to the right test.
By Hira Noor, BS Mathematics · Updated · 13 min read
Summary
To choose a statistical test, answer three questions in order. What is your goal: comparing groups, measuring a relationship, checking fit to an expected distribution, or predicting an outcome? What type is the outcome variable: continuous, categorical or ordinal? And how is the study designed: how many groups, and are the measurements independent or paired? The answers point to one test, such as a t-test, ANOVA, chi-square test, correlation or regression. A final assumption check decides between that test and its rank-based (nonparametric) alternative.
Key takeaways
- Start from the research question, not the data file: compare, relate, fit or predict.
- The type of the outcome variable (continuous, categorical or ordinal) narrows the choice more than anything else.
- Paired designs need paired tests; independent groups need independent-samples tests.
- Use one ANOVA, not a series of t-tests, for three or more groups.
- Means call for t-tests or ANOVA; counts in categories call for chi-square or proportion tests.
- Switch to a rank-based (nonparametric) test when the outcome is ordinal or small samples are clearly non-normal.
On this page
- What a statistical test does
- How to choose a statistical test in three decisions
- Decision 1: What is your goal?
- Decision 2: What type is your outcome variable?
- Decision 3: How many groups, and are they paired?
- Which statistical test to use: decision table
- t-test, ANOVA or chi-square?
- Parametric or nonparametric?
- Check the assumptions before you run the test
- Worked scenarios
- After the test: effect size and sample size
- Running the test in R, Python or SPSS
- Common mistakes when choosing a test
- Frequently asked questions
What a statistical test does
A statistical test measures how compatible your data are with a specific null hypothesis, such as “the two group means are equal” or “the two variables are unrelated.” It turns the data into a test statistic (t, F, χ², z or r), compares that statistic with the distribution it would follow if the null hypothesis were true, and reports a p-value: the probability of a result at least as extreme as yours under that null hypothesis.
Each test is built for one kind of question, one kind of data and one kind of design. Choosing the wrong test rarely produces an error message. It produces a p-value that answers a different question from the one you asked, which is why the choice matters more than the arithmetic.
The main types of statistical tests follow the four goals below: comparison tests (z-tests, t-tests, ANOVA), correlation tests (Pearson, Spearman), tests on categorical counts (chi-square, Fisher’s exact) and regression models. Each comes in a parametric form and, for most designs, a rank-based nonparametric form.
How to choose a statistical test in three decisions
Work through three decisions in order, then run one assumption check:
- 1. GoalCompare groups, measure a relationship, test fit or predict?
- 2. Outcome typeContinuous, categorical or ordinal?
- 3. DesignHow many groups? Independent or paired?
- Assumption checkParametric test, or its rank-based alternative?
- Goal. Are you comparing groups, measuring a relationship between variables, checking whether counts fit an expected distribution, or predicting an outcome?
- Outcome type. Is the variable you measured continuous (time, weight, score), categorical (yes/no, brand chosen) or ordinal (a ranking or a 1–5 rating)?
- Design. How many groups or samples are there, and are they independent (different subjects) or paired (the same subjects measured twice, or matched pairs)?
The final check decides between a standard (parametric) test and a rank-based (nonparametric) alternative: are the test’s assumptions about normality and equal spread reasonable for your data? The assumptions section below shows how to check.
Statistical test chooser: answer the questions to get a suggestion
Suggested test
Welch’s t-test
Safe default; the pooled two-sample t-test is fine when the standard deviations are similar.
Run it: Welch’s t-Test Calculator
The chooser covers the common cases in this guide. Read the sections below to see why each answer leads where it does, and check the assumptions before you report a result.
Decision 1: What is your goal?
Most questions in applied statistics fall under four goals. Write your question as one sentence first; the verb usually tells you which goal it is.
| Goal | Typical question | Test family |
|---|---|---|
| Compare groups | Do customers who saw design B spend more than those who saw design A? | z-tests, t-tests, ANOVA, proportion tests |
| Measure a relationship | Is study time associated with exam score? | Pearson or Spearman correlation; chi-square test of independence for categories |
| Test goodness of fit | Are the six faces of this die equally likely? | Chi-square goodness-of-fit test |
| Predict an outcome | How much does price rise for each extra square metre of floor space? | Simple, multiple or logistic regression |
Comparison and relationship questions overlap. “Does smoking status differ by age group?” compares proportions across groups and also asks whether two categorical variables are associated. The chi-square test of independence answers both forms of the question.
Decision 2: What type is your outcome variable?
The outcome (dependent) variable is the thing you measured and want to explain. Its type decides which summary the test compares: means for continuous (quantitative) data, proportions or counts for categorical data, and ranks for ordinal data.
| Variable type | Examples | What the test compares | Typical tests |
|---|---|---|---|
| Continuous | Weight, reaction time, revenue, test score | Means (or variances) | t-tests, z-tests, ANOVA, F-test, Pearson correlation, linear regression |
| Binary | Converted or not, pass or fail | Proportions | Proportion z-tests, chi-square, Fisher’s exact, McNemar’s test, logistic regression |
| Nominal (3+ categories) | Blood type, preferred brand | Counts per category | Chi-square goodness of fit, chi-square test of independence |
| Ordinal | Satisfaction on a 1–5 scale, finishing position | Ranks | Mann–Whitney U, Wilcoxon signed-rank, Kruskal–Wallis, Spearman correlation |
A fifth type, time-to-event data such as time until relapse or until a machine fails, needs survival methods: the log-rank test to compare groups and Cox regression to model the outcome. These handle subjects who have not yet had the event when the study ends, which ordinary t-tests cannot.
Decision 3: How many groups, and are they paired?
For comparison questions, the design picks the specific test within a family.
- One sample against a fixed value. You compare one sample’s mean or proportion with a target, a historical figure or a claim. Example: is the mean fill volume different from 500 ml? See the one-sample t-test guide.
- Two independent groups. Each subject belongs to exactly one group, and the groups do not influence each other. Example: a treatment group and a separate control group. See the independent-samples t-test guide.
- Two paired measurements. Each value in one set is linked to one value in the other: before and after on the same person, left eye and right eye, or matched case–control pairs. The analysis works on the within-pair differences. See the paired t-test guide.
- Three or more groups. Run one overall test (ANOVA for means, chi-square for proportions) and follow up with post-hoc comparisons, rather than testing every pair separately.
- Two grouping factors at once. When groups are defined by two factors, such as fertilizer and watering level, a two-way ANOVA tests each factor and their interaction. See one-way vs two-way ANOVA.
Which statistical test to use: decision table
Find the row that matches your goal, outcome and design. Where Statculator has a calculator for the test, the last column links to it; each calculator shows the full working.
| Goal and design | Outcome | Test | Calculator |
|---|---|---|---|
| One mean vs a target value, σ unknown | Continuous | One-sample t-test | One-Sample t-Test Calculator |
| One mean vs a target value, σ known | Continuous | One-sample z-test | One-Sample Z-Test Calculator |
| Two independent means, similar variances | Continuous | Two-sample (pooled) t-test | Two-Sample t-Test Calculator |
| Two independent means, unequal variances | Continuous | Welch’s t-test | Welch’s t-Test Calculator |
| Two independent groups, skewed or ranked data | Ordinal or continuous | Mann–Whitney U test | Software (see below) |
| Two paired measurements | Continuous | Paired t-test | Paired t-Test Calculator |
| Two paired measurements, skewed differences | Ordinal or continuous | Wilcoxon signed-rank test | Software (see below) |
| Three or more independent means | Continuous | One-way ANOVA | ANOVA Calculator |
| Three or more independent groups, skewed or ranked data | Ordinal or continuous | Kruskal–Wallis H test | Software (see below) |
| Two grouping factors and their interaction | Continuous | Two-way ANOVA | Two-way ANOVA guide |
| Same subjects measured three or more times | Continuous | Repeated-measures ANOVA (Friedman test if ranked) | Software (see below) |
| Two variances | Continuous | F-test for equal variances | F-Test Calculator |
| One proportion vs a claimed value | Binary | One-proportion z-test | One Proportion Z-Test Calculator |
| Two independent proportions | Binary | Two-proportion z-test | Two Proportion Z-Test Calculator |
| Two paired proportions (before and after on the same people) | Binary | McNemar’s test | Software (see below) |
| Association between two categorical variables | Categorical | Chi-square test of independence | Chi-Square Test of Independence Calculator |
| 2×2 table with small expected counts | Binary | Fisher’s exact test | Fisher’s Exact Test Calculator |
| Observed counts vs an expected distribution | Categorical | Chi-square goodness of fit | Chi-Square Goodness of Fit Calculator |
| Linear relationship between two variables | Continuous | Pearson correlation test | Pearson’s Correlation Test Calculator |
| Monotonic relationship, ranked data or outliers | Ordinal or continuous | Spearman rank correlation | Spearman Correlation Calculator |
| Predict an outcome from one predictor | Continuous | Simple linear regression | Linear Regression Calculator |
| Predict an outcome from several predictors | Continuous | Multiple linear regression | Multiple Linear Regression Calculator |
| Predict a yes/no outcome | Binary | Logistic regression | Logistic Regression Calculator |
Two rules sit behind several rows. Use a z-test for a mean only when the population standard deviation is genuinely known; otherwise use t. And a chi-square test needs adequate expected counts, commonly at least 5 per cell; when a 2×2 table falls short, switch to Fisher’s exact test.
Try it: T-Test CalculatorNot sure which t-test fits your design? This calculator runs one-sample, pooled, Welch and paired t-tests from the same screen.t-test, ANOVA or chi-square?
These three tests cover most introductory comparisons, and the choice between them comes down to two facts: what the outcome is, and how many groups you have. A t-test and ANOVA compare means of a continuous outcome. A chi-square test compares counts of a categorical outcome.
| Feature | t-test | One-way ANOVA | Chi-square test of independence |
|---|---|---|---|
| Outcome | Continuous | Continuous | Categorical |
| Groups compared | One or two | Three or more | Two or more categories of each variable |
| What it compares | Means | Means | Observed vs expected counts |
| Test statistic | t | F | χ² |
| Example question | Do two diets give different mean weight loss? | Do four fertilizers give different mean yields? | Is preferred brand related to age group? |
A quick way to tell them apart is to look at one row of your data. If the outcome cell holds a number you could average, such as 72.4 kg, use a t-test or ANOVA. If it holds a label, such as “Brand B,” and your analysis counts how often each label appears, use chi-square. Converting a continuous outcome into categories just to run a chi-square test throws away information and usually loses power.
Parametric or nonparametric?
Parametric tests such as the t-test and ANOVA assume that the data, or more precisely the sampling distribution of the mean, are approximately normal, and some also assume equal variances. Nonparametric tests replace the values with their ranks, so they need weaker assumptions and are less affected by outliers. The cost is some loss of power when the parametric assumptions do hold, and a slightly different hypothesis: rank tests compare whole distributions or typical values rather than means.
| Parametric test | Nonparametric alternative | Design |
|---|---|---|
| One-sample t-test | Wilcoxon signed-rank test | One sample vs a value |
| Paired t-test | Wilcoxon signed-rank test on the differences | Two paired measurements |
| Two-sample or Welch’s t-test | Mann–Whitney U (Wilcoxon rank-sum) test | Two independent groups |
| One-way ANOVA | Kruskal–Wallis H test | Three or more independent groups |
| Repeated-measures ANOVA | Friedman test | Three or more related measurements |
| Pearson correlation | Spearman rank correlation | Two variables |
Prefer the nonparametric version when the outcome is ordinal, when groups are small (roughly under 15–20 each) and clearly skewed, or when a few extreme values dominate. With moderate or large samples, t-tests and ANOVA hold up well against non-normality because of the central limit theorem, so mild skew alone is not a reason to switch.
Check the assumptions before you run the test
Once the three decisions point to a test, check that its assumptions are reasonable. Do this by looking at the data, not by trying several tests and keeping the one with the smallest p-value. In formal research, name the main test and its fallback in the protocol or statistical analysis plan before the data are analysed.
| Assumption | Applies to | How to check | If it fails |
|---|---|---|---|
| Independent observations | Almost every test | Study design: one value per subject, no clustering | Use a paired, repeated-measures or multilevel method |
| Approximate normality | t-tests, ANOVA, Pearson test | Plot a histogram or box plot; for small samples, a Shapiro–Wilk test | Use the rank-based alternative, or rely on a large sample |
| Equal variances | Pooled t-test, one-way ANOVA | Compare standard deviations; run Levene’s test | Use Welch’s t-test or Welch’s ANOVA |
| Expected counts of at least 5 | Chi-square tests | Compute expected counts for every cell | Fisher’s exact test (2×2) or combine sparse categories |
| Linear relationship | Pearson correlation, linear regression | Draw a scatter plot | Spearman correlation or a curved model |
Worked scenarios
Each scenario applies the three decisions in order.
Blood pressure before and after a program
Problem. A clinic measures systolic blood pressure for 25 patients before and after a 12-week exercise program. Did blood pressure change?
- 01Goal: compare two sets of measurements.
- 02Outcome: blood pressure in mmHg, a continuous variable.
- 03Design: the same 25 patients are measured twice, so the sets are paired.
- 04Assumption check: plot the 25 differences (after − before). If they are roughly symmetric with no extreme outliers, a t-test is appropriate.
Answer. Use a paired t-test on the differences (Paired t-Test Calculator). If the differences are strongly skewed, use the Wilcoxon signed-rank test instead.
Landing-page conversion
Problem. An online store shows page A to 2,000 visitors and page B to a different 2,000 visitors, and records whether each visitor bought something. Do the conversion rates differ?
- 01Goal: compare two groups.
- 02Outcome: bought or did not buy, a binary variable, so the summary is a proportion.
- 03Design: different visitors saw each page, so the groups are independent.
- 04Sample size: thousands of visitors per page keep expected counts far above 5.
Answer. Use a two-proportion z-test (Two Proportion Z-Test Calculator). A chi-square test on the 2×2 table without continuity correction gives the same two-sided p-value, because χ² = z² for this design.
Four fertilizers
Problem. An agronomist applies four fertilizers to six plots each and records wheat yield in tonnes per hectare. Does yield depend on fertilizer?
- 01Goal: compare groups.
- 02Outcome: yield, a continuous variable.
- 03Design: four independent groups, so a two-group t-test is not enough.
- 04Follow-up: if the overall test is significant, compare pairs with a post-hoc method such as Tukey’s HSD.
Answer. Use a one-way ANOVA (ANOVA Calculator) with df = 3 and 20. The one-way ANOVA guide works through the full calculation.
Satisfaction ratings at three branches
Problem. A bank asks 12 customers at each of three branches to rate their visit from 1 (very poor) to 5 (excellent). Most ratings are 4 or 5. Do satisfaction levels differ between branches?
- 01Goal: compare groups.
- 02Outcome: a single 1–5 rating, an ordinal variable.
- 03Design: three independent groups of different customers.
- 04Assumption check: with 12 ratings per branch piled up at 4 and 5, the data are ordinal and skewed, so a test on ranks fits better than one on means.
Answer. Use a Kruskal–Wallis H test, the rank-based counterpart of one-way ANOVA, with df = 3 − 1 = 2. If it is significant, follow up with pairwise rank tests and a correction for multiple comparisons.
Sleep and exam scores
Problem. A lecturer records hours of sleep the night before an exam and the exam score for 40 students. Are the two related, and can sleep predict the score?
- 01Goal: first a relationship, then a prediction.
- 02Variables: both continuous.
- 03Plot first: a scatter plot shows whether the pattern is roughly linear and whether a few points drive it.
- 04If the pattern is linear, test the correlation, then fit a line to predict score from sleep.
Answer. Use a Pearson correlation test (Pearson’s Correlation Test Calculator) for the association and simple linear regression (Linear Regression Calculator) for prediction. If the pattern is curved but consistently rising, use Spearman’s correlation.
Is the die fair?
Problem. A die is rolled 120 times and the count for each face is recorded. Is every face equally likely?
- 01Goal: compare observed counts with an expected distribution.
- 02Outcome: the face shown, a categorical variable with six levels.
- 03Expected counts: 120 / 6 = 20 per face, comfortably above 5.
Answer. Use a chi-square goodness-of-fit test (Chi-Square Goodness of Fit Calculator) with df = 6 − 1 = 5.
After the test: effect size and sample size
A p-value says whether an effect is distinguishable from zero, not whether it is large. Report an effect size next to every test: Cohen’s d for a difference in means, η² for ANOVA, Cramér’s V for a chi-square table, or r itself for a correlation. The Effect Size Calculator computes the common ones.
The same choices also shape planning. A paired design or a parametric test usually needs fewer subjects than the alternative to detect the same effect, and the significance level and power you choose set the risk of a false positive or a missed effect; see Type I vs Type II errors. Use the Sample Size Calculator to size a study before collecting data.
Running the test in R, Python or SPSS
Once you know the test, every major package runs it in one command or menu. The names below are the standard functions; check each one’s defaults, such as whether a t-test assumes equal variances or whether a continuity correction is applied.
| Test | R | Python (scipy.stats) | SPSS menu |
|---|---|---|---|
| One-sample, Welch or paired t-test | t.test() | ttest_1samp, ttest_ind, ttest_rel | Analyze → Compare Means |
| One-way ANOVA | aov() or oneway.test() | f_oneway | Analyze → Compare Means → One-Way ANOVA |
| Mann–Whitney U, Wilcoxon signed-rank | wilcox.test() | mannwhitneyu, wilcoxon | Analyze → Nonparametric Tests |
| Kruskal–Wallis | kruskal.test() | kruskal | Analyze → Nonparametric Tests |
| Chi-square test of independence | chisq.test() | chi2_contingency | Analyze → Descriptive Statistics → Crosstabs |
| Fisher’s exact test | fisher.test() | fisher_exact | Crosstabs → Exact |
| Pearson or Spearman correlation | cor.test() | pearsonr, spearmanr | Analyze → Correlate → Bivariate |
| McNemar’s test | mcnemar.test() | statsmodels: mcnemar | Crosstabs → Statistics → McNemar |
| Friedman test | friedman.test() | friedmanchisquare | Analyze → Nonparametric Tests |
R’s t.test() runs Welch’s test unless you set var.equal = TRUE, and its chisq.test() applies Yates’ continuity correction to 2×2 tables by default. Those defaults explain most small differences between software output and a hand calculation.
Common mistakes when choosing a test
- Running many t-tests instead of one ANOVA. Three groups give three pairwise comparisons. If they were independent tests at α = 0.05, the chance of at least one false positive would be 1 − 0.95³ ≈ 0.143, nearly three times the intended rate.
- Ignoring pairing. Before-and-after data analysed with an independent-samples test usually loses power, sometimes dramatically.
- Choosing the tail after seeing the data. Decide between a one-tailed and a two-tailed test before looking at the results. Switching to one tail because the effect went the “right” way halves the p-value without justification; see What Is a P-Value? Meaning, Calculation and Common Mistakes.
- Using Pearson’s r for a curved relationship. Pearson’s r measures linear association only, and a strong U-shaped pattern can give r close to 0. Plot the data first.
- Applying a chi-square test to sparse tables. When expected counts fall below 5, the chi-square approximation becomes unreliable. Fisher’s exact test avoids the problem for 2×2 tables.
- Running a chi-square test on means or percentages. Chi-square works on raw counts. Feeding it averages or percentages gives a meaningless statistic.
- Testing a whole population. If you have data on every member of the group you care about, there is no sampling error to test. Describe the data instead; see Descriptive vs Inferential Statistics: Differences and Examples.
Frequently asked questions
What does a statistical test do?
What are the main types of statistical tests?
Should I use ANOVA or chi-square?
How do I know when to use a t-test or a chi-square test?
Should I use a t-test or a z-test?
When should I use ANOVA instead of a t-test?
Which test should I use for categorical data?
Do I need normally distributed data for a t-test?
What is the difference between correlation and regression?
Can I choose a test after looking at my data?
Further reading
- NIST/SEMATECH e-Handbook of Statistical Methods, Section 7.1.3: What are statistical tests?
- NIST/SEMATECH e-Handbook of Statistical Methods, Section 7.4.3: Are the means equal?
- UCLA Statistical Methods and Data Analytics, Choosing the correct statistical test in SAS, Stata, SPSS and R
- Parab S, Bhalerao S. Choosing statistical test. International Journal of Ayurveda Research, 2010;1(3):187–191
- OpenStax, Introductory Statistics 2e
Cite this page
Statculator. (2026, November 14). How to Choose the Right Statistical Test. Retrieved November 14, 2026, from https://statculator.com/learn/how-to-choose-a-statistical-test/