You reject the null hypothesis in a chi-square test when your p-value is less than your chosen significance level (alpha), typically 0.05. Equivalently, you reject it when your calculated chi-square statistic exceeds the critical value from the chi-square distribution table for your degrees of freedom. Both methods give you the same answer, just from different angles.
The Two Ways to Make the Decision
There are two paths to the same conclusion, and which one you use depends on whether you’re working with software or a printed table.
The p-value method: After computing your chi-square statistic, you get a p-value, which represents the probability of seeing results at least this extreme if the null hypothesis were true. If that p-value falls below your alpha level (usually 0.05), you reject the null. A p-value of 0.03, for example, means there’s only a 3% chance your data would look this way if there were truly no relationship between the variables. Since 0.03 is less than 0.05, you reject the null.
The critical value method: You look up the critical value in a chi-square table using your degrees of freedom and your alpha level. If your calculated chi-square statistic is larger than that critical value, you reject the null hypothesis. The chi-square test is almost always a right-tailed (upper-tail) test, so you’re checking whether your statistic lands far enough to the right of the distribution to be considered unlikely under the null hypothesis.
How Degrees of Freedom Affect the Threshold
The critical value you need to beat depends on your degrees of freedom, which reflect the size of the data you’re analyzing. For a chi-square test of independence (the most common type), degrees of freedom are calculated as (rows − 1) × (columns − 1). A simple 2×2 table has 1 degree of freedom. A 3×4 table has 6.
For a goodness-of-fit test, where you’re checking whether observed data matches an expected distribution, degrees of freedom equal the number of categories minus 1. If you’re testing whether a die is fair across its 6 faces, you have 5 degrees of freedom.
This matters because the critical value rises as degrees of freedom increase. At alpha = 0.05 with 1 degree of freedom, the critical value is about 3.84. With 5 degrees of freedom, it jumps to about 11.07. A larger table needs a larger chi-square statistic to reach significance because there are more cells contributing to the overall value.
What the Null Hypothesis Actually Claims
In a test of independence, the null hypothesis states that the two variables in your contingency table are unrelated. If you’re comparing vaccination status and disease outcome, the null says vaccination has no effect on whether someone gets the disease. Rejecting the null means you have statistical evidence that the two variables are associated.
In a goodness-of-fit test, the null hypothesis claims that your observed data follows a specific expected distribution. Rejecting it means your data deviates from that expected pattern more than random chance would explain.
Choosing Your Significance Level
The alpha level is the threshold you set before running the test. It represents the maximum risk of a false positive you’re willing to accept. At alpha = 0.05, you’re accepting a 5% chance of incorrectly rejecting a null hypothesis that’s actually true. Most biomedical and social science research uses 0.05 as the standard. Some fields or high-stakes studies use 0.01, which requires stronger evidence but increases the risk of missing a real effect.
Your alpha level is also, by definition, the probability of committing a Type I error: concluding that a relationship exists when it doesn’t. Think of it like convicting an innocent person. A Type II error goes the other direction: failing to detect a relationship that genuinely exists, like letting a guilty person go free. Increasing your sample size reduces the likelihood of both types of errors.
When the Chi-Square Test Can Mislead You
The chi-square test assumes certain conditions are met, and violating them can produce unreliable results. The most important rule involves expected frequencies: the counts your table would show if the null hypothesis were true. When any cell in your table has an expected count below 5, the standard chi-square approximation becomes less accurate.
For 2×2 tables with small expected counts, Yates’s correction helps. It subtracts 0.5 from the absolute difference between observed and expected values in each cell before squaring, which produces a more conservative (slightly larger) p-value. With large samples, this correction barely changes the result. For very small samples where multiple cells have low expected counts, Fisher’s exact test is a better alternative than chi-square altogether.
A Significant Result Is Not the Whole Story
Rejecting the null hypothesis tells you that a relationship likely exists, but it says nothing about how strong that relationship is. A massive sample can produce a statistically significant result from a trivially small difference. This is where effect size comes in.
For a 2×2 table, you can calculate a measure called phi, which ranges from 0 (no association) to 1 (perfect association). For larger tables, Cramer’s V serves the same purpose. It’s calculated by dividing the chi-square statistic by the sample size and the smaller dimension of the table minus 1, then taking the square root. A Cramer’s V of 0.1 suggests a weak association, 0.3 moderate, and 0.5 or above strong. Reporting effect size alongside your p-value gives a much clearer picture of whether your finding is practically meaningful, not just statistically detectable.
Putting It Together: A Quick Decision Checklist
- Set alpha before testing (0.05 is standard).
- Calculate degrees of freedom using (rows − 1) × (columns − 1) for independence tests, or categories − 1 for goodness-of-fit.
- Check expected frequencies. If any cell is below 5, apply Yates’s correction for a 2×2 table or use Fisher’s exact test.
- Compare your result. If your p-value is less than alpha, or your chi-square statistic exceeds the critical value, reject the null hypothesis.
- Report effect size. Use phi for 2×2 tables or Cramer’s V for larger tables to show how strong the association is.
If your p-value is greater than alpha, you fail to reject the null. This doesn’t prove the variables are unrelated. It means your data didn’t provide enough evidence to conclude otherwise, given your sample size and the effect you were looking for.