Statistics in research is the set of mathematical tools used to collect, organize, analyze, and interpret numerical data so researchers can draw meaningful conclusions. It transforms raw numbers into evidence, letting researchers move from “here’s what we observed” to “here’s what it means.” Without statistical analysis, research findings would be little more than educated guesses.
Descriptive vs. Inferential Statistics
Statistics in research falls into two broad categories, and understanding the difference between them is essential to reading or conducting any study.
Descriptive statistics summarize what the data look like. They answer the question: what happened in this particular group of people or observations? The two main tools here are measures of central tendency (the “average”) and measures of dispersion (how spread out the data are). Central tendency includes the mean (add up all values and divide by the count), the median (the middle value when data are ranked), and the mode (the most frequently occurring value). Dispersion includes the range (difference between the highest and lowest values), the interquartile range (the span of the middle 50% of values), and the standard deviation (how far, on average, each data point sits from the mean). Knowing the average alone is limited. Two classrooms could have the same average test score, but in one room every student scored close to 75, while in the other scores ranged from 30 to 100. Dispersion captures that difference.
Inferential statistics go a step further. They use data from a smaller group (the sample) to make predictions or generalizations about a larger group (the population). Probability is the underlying concept that links a sample to the population it came from. If you survey 1,000 adults about sleep habits, inferential statistics let you estimate what all adults in the country might report, along with a measure of how confident you can be in that estimate.
How Hypothesis Testing Works
Most quantitative research revolves around hypothesis testing, a structured process for deciding whether observed results reflect a real pattern or just random chance. The starting point is the null hypothesis, which assumes there is no effect, no difference, or no relationship between the variables being studied. The alternative hypothesis is the researcher’s actual prediction: that a difference or effect does exist.
The core question hypothesis testing asks is: if the null hypothesis were true and nothing interesting were happening, how likely would we be to see data this extreme? Researchers choose a statistical test, calculate a test statistic from the data, and then determine whether the result falls in a region extreme enough to reject the null hypothesis. The output is a p-value, a number that quantifies how surprising the observed data would be if nothing were really going on.
What P-Values Actually Tell You
By long-standing convention, a p-value below 0.05 is considered “statistically significant,” meaning there is less than a 5% probability the observed result would occur by chance alone if the null hypothesis were true. But this threshold is not a magic line between real and fake findings. In 2016, the American Statistical Association released a formal statement warning against its misuse, with several key points: a statistically significant p-value does not prove an effect exists, a non-significant p-value does not prove an effect is absent, and a p-value does not measure the probability that your hypothesis is true.
Some researchers have proposed lowering the threshold to 0.005 to reduce false positives, though others argue this would create new problems, particularly for smaller, independently funded studies that can’t easily increase their sample sizes. The current best practice is to report exact p-values as continuous numbers rather than simply labeling results as “significant” or “not significant.”
Type I and Type II Errors
Statistical testing can go wrong in two directions. A Type I error (false positive) happens when a researcher concludes there is an effect or association, but none actually exists in the population. Think of it like convicting an innocent person. The probability of making this error is set by the significance threshold, typically 0.05 or 0.01.
A Type II error (false negative) is the opposite: the researcher concludes there is no effect when one truly exists. This is like letting a guilty person go free. The most common cause of Type II errors is a sample that’s too small, especially when the real effect is modest in size. Both errors carry consequences. In medical research, a false positive might lead to adopting a treatment that doesn’t work, while a false negative might cause researchers to abandon a treatment that could have helped.
Why Sample Size and Power Matter
Statistical power is the probability that a study will correctly detect a real effect when one exists. The ideal power for a study is 0.80, meaning an 80% chance of catching a true effect. Three factors drive power: sample size, the size of the effect you’re looking for, and the significance threshold you set.
Small samples paired with small effects are a recipe for wasted effort. The study simply won’t have enough data to distinguish a real pattern from noise. Increasing the sample size is the most straightforward way to boost power, but it also raises costs and extends timelines. On the other hand, very large samples introduce their own problem: they can make trivially small differences appear statistically significant, even when those differences have no practical importance. A study with 50,000 participants might find a “significant” blood pressure reduction of 0.5 points, a number too small to matter for any individual patient. The confidence interval, which gives a range of plausible values for the true effect, also narrows as sample size increases, providing more precise estimates.
Choosing the Right Statistical Test
Researchers select a statistical test based on the type of data they have and the question they’re asking. Several common tests come up repeatedly across disciplines:
- T-test: Compares the averages of two groups. An independent samples t-test compares two separate groups (for example, a treatment group vs. a control group), while a paired t-test compares the same group at two time points (before and after an intervention). The data need to be continuous and roughly follow a bell-shaped distribution.
- ANOVA (analysis of variance): An extension of the t-test for three or more groups. If you’re comparing pain levels across four different medications, ANOVA tells you whether at least one group differs from the others. Two-way ANOVA adds a second grouping variable, letting you examine the effects of two factors simultaneously.
- Regression: Examines how one variable changes in relation to another. It produces a coefficient showing the direction and strength of the relationship, along with a confidence interval for that estimate.
All of these tests require the dependent variable to be measured on a continuous scale. Choosing the wrong test for your data type can produce misleading results, which is why study design and statistical planning happen before data collection, not after.
The Role of Sampling
How participants or observations are selected into a study determines whether the results can be generalized to a larger population. Two broad approaches exist: probability sampling and non-probability sampling.
In probability sampling, every individual in the target population has a known, equal chance of being selected. This is the only type of sampling that allows researchers to draw conclusions about an entire population. Random selection also protects against researcher bias in choosing who gets included. A sample will likely represent its target population if two conditions are met: it’s large enough, and it was formed using a random technique.
Non-probability sampling, such as convenience sampling (recruiting whoever is easiest to reach) or purposeful sampling (using the researcher’s judgment to hand-pick participants), cannot be assumed to represent the broader population. The results apply only to the people actually studied. Purposeful sampling is common in qualitative research, where depth of understanding matters more than generalizability, but in quantitative work it limits what conclusions you can draw.
Statistical Significance vs. Practical Significance
One of the most important distinctions in research is the gap between a result that is statistically significant and one that actually matters in the real world. Statistical significance tells you whether a mathematical pattern exists in the data. Practical (or clinical) significance tells you whether that pattern is large enough to change decisions, improve outcomes, or justify costs.
Consider two cancer drugs tested in separate trials, both producing statistically significant improvements in survival compared to standard treatment. Drug A extends survival by five years. Drug B extends it by five months. Both results pass the p-value threshold, but their clinical value is vastly different. Because sample size and measurement variability can easily influence statistical results, a non-significant outcome doesn’t necessarily mean a treatment is useless, and a significant outcome doesn’t guarantee it’s worth pursuing.
Researchers are increasingly expected to report effect sizes and confidence intervals alongside p-values, giving readers the information they need to judge whether a finding has real-world weight. A complete picture of any research result requires all three: the p-value to assess chance, the effect size to assess magnitude, and the confidence interval to assess precision.