There is no single number that qualifies as a “statistically significant sample size.” The sample size you need depends on four interconnected factors: how confident you want to be in your results, how precise you need them to be, how large the effect you’re measuring is, and how much variation exists in your population. For a common scenario like a general population survey with 95% confidence and a 5% margin of error, you need roughly 384 people once your population exceeds about 100,000. But that number shifts dramatically depending on what you’re studying and how you’re studying it.
The confusion behind this search is understandable. People often treat “statistically significant” as a property of the sample size itself, when it’s actually a property of the result. A sample size doesn’t make your findings significant. It makes your study powerful enough to detect a real effect if one exists. The goal is to recruit enough people (or collect enough data points) so that your study has a fair shot at finding what you’re looking for.
What Statistical Significance Actually Means
Statistical significance is a way of asking: could this result have happened by chance alone? The standard threshold is a p-value of 0.05, meaning there’s less than a 5% probability that the observed result would occur if nothing real were going on. Ronald Fisher, the statistician who popularized this cutoff, originally described it as “convenient” rather than sacred. For decades, though, researchers have treated 0.05 as a hard line separating real findings from noise.
That convention is increasingly questioned. In 2016, the American Statistical Association released a formal warning against misusing p-values, stating that researchers should not base conclusions solely on whether a result crosses the 0.05 threshold. Some statisticians have proposed lowering the bar to 0.005 for stronger evidence. Others argue the threshold should stay but be interpreted more carefully. The core point: statistical significance is not proof that something is true. It’s one piece of evidence, and its strength depends heavily on how the study was designed, including its sample size.
The Four Factors That Determine Sample Size
Calculating the right sample size is called a “power analysis,” and it balances four variables. Change any one of them and the required number of participants changes too.
- Significance level (alpha): The risk you’re willing to accept of finding something that isn’t really there. The standard is 0.05, or 5%. Lowering it to 0.01 means you need more data to reach the stricter bar.
- Power: The probability that your study will detect a real effect when one exists. The widely accepted standard is 80%, meaning you accept a 20% chance of missing a real finding. Some fields push for 90%.
- Effect size: How big the difference or relationship you’re trying to detect actually is. A large effect (think: a drug that doubles survival time) is easy to spot with fewer people. A small effect (a drug that improves outcomes by 2%) requires many more.
- Variability: How much your data naturally spreads out. If people’s responses to a treatment vary wildly, you need a bigger sample to see through the noise.
These four factors work like a seesaw. If you want higher confidence, you need more participants. If the effect you’re measuring is subtle, you need more participants. There’s no way around the tradeoff.
How Effect Size Changes the Numbers
Effect size is the factor most people underestimate. The psychologist Jacob Cohen classified effect sizes into three categories: small (0.2), medium (0.5), and large (0.8 or higher). These numbers describe the magnitude of a difference in standardized terms.
Here’s why this matters practically. If you’re comparing two groups and expect a medium effect size of 0.5, you need roughly 60 people per group to reach 80% power at a significance level of 0.05. That’s 120 total. But if your expected effect is small (0.2), you might need several hundred per group. And if the effect is large (0.8), you could get away with 20 to 30 per group. The same study design can require anywhere from 50 to 800 participants depending entirely on how strong the signal is.
This is why blanket rules like “you need at least 30” or “you need at least 100” are misleading. Those numbers might work for large effects or rough estimates, but they’ll leave you underpowered for detecting smaller, subtler patterns.
Sample Sizes for Surveys
Surveys are the context where concrete benchmarks are most useful, because the math is more standardized. The key formula, developed by the statistician William Cochran, calculates sample size based on your desired confidence level, margin of error, and the estimated proportion of the trait you’re measuring in the population.
At 95% confidence with a 5% margin of error, the numbers look like this:
- Population of 100: You need about 80 responses.
- Population of 500: About 217.
- Population of 1,000: About 278.
- Population of 10,000: About 370.
- Population of 100,000 or more: About 383 to 384.
Notice the pattern: once your population gets large, the required sample size plateaus. Surveying a city of 100,000 and surveying an entire country of 300 million require nearly the same sample, around 384, for those same confidence parameters. That’s because the math depends more on the precision you want than on the total population.
Tighten the margin of error to 2.5%, and those numbers jump. A population of 10,000 now requires about 1,332 responses. Drop it to 1% and you’re looking at nearly 5,000. Precision is expensive.
Pilot Studies Use Different Rules
Not every study needs a full power analysis. Pilot studies, which test whether a larger study is feasible, follow rougher guidelines. Some researchers recommend at least 30 participants per group, while others suggest as few as 12 per group. The purpose isn’t to prove a hypothesis but to identify problems with recruitment, dropout rates, or study design before committing to a full-scale trial.
In practice, pilot studies often inflate their numbers to account for dropout. One study of paramedics, for example, started with 30 per group based on the research question but recruited 50 per group after estimating that 25% would drop out and 17% would have insufficient real-world experience to contribute useful data.
Why Bigger Isn’t Always Better
It’s tempting to think that more data is always an improvement. It’s not. An overpowered study, one with far more participants than necessary, can detect differences so tiny they have no real-world meaning. This is the gap between statistical significance and practical significance.
Consider two cancer drugs tested in separate studies. Both produce statistically significant results with a p-value of 0.01. But Drug A extends survival by five years, while Drug B extends it by five months. Both results cross the significance threshold, yet only one represents a meaningful improvement for patients. With a large enough sample, you can make almost any trivial difference look statistically significant.
Overpowered studies also waste resources. In medical research involving human subjects, enrolling far more participants than needed exposes extra people to experimental treatments or procedures without scientific justification. The goal is to recruit enough people to answer the question reliably, not to pile on data for its own sake.
How to Calculate Your Sample Size
If you’re planning a study or survey, you don’t need to do this math by hand. G*Power is a free software tool widely used in academic research. It supports sample size calculations for most common statistical tests, including comparisons between two groups, comparisons across multiple groups, correlation analyses, and tests of proportions. You select your test type, plug in your expected effect size, desired power (typically 0.80), and significance level (typically 0.05), and it returns the minimum sample size.
For surveys, online sample size calculators from sites like Qualtrics, SurveyMonkey, or Raosoft let you enter your population size, confidence level, and margin of error to get a number in seconds. These are based on the Cochran formula and work well for straightforward proportion-based surveys.
The harder part isn’t the calculation itself. It’s choosing realistic inputs. Overestimating your expected effect size is the most common mistake, because it produces a sample size that looks manageable but leaves your study unable to detect the actual, smaller effect. When in doubt, use a smaller effect size estimate. You’ll need more participants, but you’ll be far less likely to run an inconclusive study.