The research hypothesis in ANOVA is that at least one group mean differs from the others. This is the alternative hypothesis, and it stands in contrast to the null hypothesis, which states that all group means are equal. Understanding exactly how these hypotheses work, and what ANOVA can and cannot tell you, is essential for interpreting results correctly.
The Null and Alternative Hypotheses
ANOVA (analysis of variance) is built around testing two competing claims. The null hypothesis says that the population means of all groups being compared are identical. For three groups, that looks like this: the mean of group 1 equals the mean of group 2 equals the mean of group 3.
The alternative hypothesis is where most people get tripped up. It does not say that all group means are different from each other. Instead, it says that at least one group mean is different from at least one other. In formal terms, the alternative is that not all of the means are equal. This is a crucial distinction. If you’re comparing four treatment groups, rejecting the null hypothesis could mean that only one group differs while the other three are identical. It could also mean all four differ. ANOVA alone won’t tell you which scenario is true.
Why ANOVA Is Called an Omnibus Test
ANOVA is an omnibus test, meaning it evaluates all groups simultaneously in a single step rather than comparing them pair by pair. The F-test, which is the statistical engine behind ANOVA, produces a test statistic and a corresponding p-value. If that p-value falls below your chosen significance level (typically 0.05), you reject the null hypothesis and conclude that at least two groups have different means.
If the omnibus test is not significant, you stop there. The conclusion is simply that there’s no evidence of different means across the groups. You would not go on to compare individual pairs. If the test is significant, you’ve answered only the broad question: something differs. Identifying exactly which groups differ from which requires a second step.
What Happens After You Reject the Null
Because the alternative hypothesis only tells you that some difference exists somewhere, a significant ANOVA result is a starting point, not an endpoint. Researchers then use post-hoc comparison procedures to test every pair of groups and determine where the specific differences lie.
The most common of these is Tukey’s Honestly Significant Difference test, which compares all possible pairs while controlling the overall chance of a false positive. Other options include the Bonferroni correction, which divides the acceptable error rate by the number of comparisons being made, and Dunnett’s test, which is designed for situations where you only care about comparing each group to a single control group. The choice depends on the study design, but all of them exist for the same reason: ANOVA’s hypothesis is deliberately broad, so follow-up tests are needed to get specific answers.
Hypotheses in Two-Way ANOVA
When a study involves two grouping variables instead of one, a two-way ANOVA tests three separate null hypotheses. The first says that the means are equal across the levels of the first variable. The second says the same for the second variable. The third, and often the most interesting, tests whether there is an interaction between the two variables.
An interaction effect means that the influence of one variable depends on the level of the other. For example, if you’re studying how both exercise type and diet affect weight loss, an interaction would mean that the best diet changes depending on which exercise someone does. The null hypothesis for the interaction states that no such dependency exists. If rejected, it tells you the two variables don’t act independently, which often reshapes how you interpret the main effects.
Hypotheses in Repeated Measures ANOVA
When the same participants are measured multiple times (before treatment, during, and after, for instance), the null hypothesis stays conceptually the same: all time-point means are equal. The alternative is that at least one time point differs. However, repeated measures designs introduce an additional technical requirement called sphericity, which assumes that the variability between all pairs of time points is roughly equal.
When sphericity is violated, the F-test becomes too liberal, meaning it rejects the null hypothesis more often than it should. This inflates false positive rates. Researchers typically check for this using Mauchly’s test, and if the assumption is violated, they apply corrections that adjust the degrees of freedom downward to make the test more conservative. The core hypothesis doesn’t change, but the path to testing it does.
Assumptions That Must Hold
The ANOVA hypothesis test is only valid when certain conditions are met. The observations within each group need to be independent of one another. The data in each group should follow an approximately normal distribution. And the variability (variance) within each group should be roughly similar across groups. When these assumptions break down, the p-value you get from the F-test may not accurately reflect the true probability of seeing your results under the null hypothesis, which undermines the entire testing framework.
Beyond Significance: Effect Size
Rejecting the null hypothesis tells you that a difference exists, but it says nothing about how large or meaningful that difference is. A study with thousands of participants can produce a statistically significant result from a trivially small difference. This is why researchers also report effect sizes alongside their ANOVA results.
The most commonly reported effect size for ANOVA is eta-squared, which represents the proportion of total variability in the data that can be attributed to group differences. A related measure called omega-squared is considered less biased, though it appears in published research less frequently. Both give you a practical sense of whether the group differences are large enough to matter, not just large enough to be detected.
How ANOVA Results Are Reported
Under APA formatting guidelines, ANOVA results are reported with the F-statistic rounded to two decimal places, accompanied by degrees of freedom (which reflect the number of groups and total observations) and the exact p-value to two or three decimal places. A p-value smaller than .001 is simply reported as “p < .001" rather than writing out all the zeros. You'll typically see something like F(3, 120) = 4.57, p = .004, which tells readers the test had four groups, 121 total observations, a moderately large F-value, and a result that crossed the significance threshold.
The degrees of freedom are important context because they indicate the study’s structure and sample size, both of which affect how much weight to give the finding. A significant result with very large degrees of freedom (meaning a very large sample) could reflect a difference too small to be practically useful, which circles back to why effect sizes matter.