A result that is “not statistically significant” means the data from a study didn’t provide strong enough evidence to conclude that a real effect or difference exists. It does not mean the study proved nothing is going on. It means the observed result could plausibly be explained by chance alone, so researchers can’t confidently rule out that possibility.
This distinction trips up a lot of people, including scientists themselves. Understanding what a non-significant result actually tells you (and what it doesn’t) can change how you read headlines about health, nutrition, and just about any study that involves numbers.
The Basic Logic Behind Significance
Statistical testing works by starting with an assumption called the null hypothesis. The null hypothesis is essentially the boring explanation: there’s no real difference between two groups, no real effect from a treatment, no real relationship between two variables. Researchers then collect data and ask, “How likely would we be to see results like these if the null hypothesis were true?”
That likelihood is expressed as a p-value, a number between 0 and 1. A small p-value means the data would be very unlikely under the null hypothesis, which gives researchers confidence that something real is happening. A large p-value means the data are perfectly compatible with the null hypothesis, so there’s no strong reason to reject it.
Before running the study, researchers set a cutoff called the alpha level. In most fields, this is 0.05, meaning they want less than a 5% chance of being wrong if they declare a result significant. If the p-value comes in below 0.05, the result is “statistically significant.” If it lands at 0.05 or above, the result is “not statistically significant,” and the researchers cannot reject the null hypothesis.
Why Non-Significant Doesn’t Mean “No Effect”
This is the single most important thing to understand. A non-significant result does not prove that a treatment doesn’t work, that two groups are identical, or that a relationship doesn’t exist. It only means the study didn’t find strong enough evidence to say otherwise. As a widely cited paper in The BMJ put it: absence of evidence is not evidence of absence.
Think of it like a courtroom. A “not guilty” verdict doesn’t mean the defendant is innocent. It means the prosecution didn’t present enough evidence to meet the standard of “beyond a reasonable doubt.” In statistics, a non-significant p-value is the equivalent of “not guilty.” The study simply didn’t meet its burden of proof. The real effect might be there but too small for the study to detect, or the study might not have enrolled enough participants to pick it up reliably.
Why a Study Might Miss a Real Effect
Several factors can cause a genuinely effective treatment or a real relationship to produce a non-significant result.
- Sample size too small. A study with 30 participants has much less ability to detect a subtle difference than one with 3,000 participants. Small studies produce noisy data, and real effects can easily get lost in that noise.
- The effect is small. If a new drug lowers blood pressure by 2 points instead of 20, you need a much larger and more precise study to distinguish that small change from random variation.
- High variability. When individual responses vary wildly (some people improve a lot, others not at all), the average effect gets harder to pin down statistically, even if the treatment genuinely helps a subset of people.
The probability of missing a real effect is called a Type II error, or a false negative. Researchers aim for statistical power of at least 80%, meaning they want at least an 80% chance of detecting a real effect if one exists. But many published studies fall short of that threshold, which means non-significant findings are common even when the treatment or relationship under investigation is real.
The 0.05 Threshold Isn’t Universal
The 0.05 cutoff is a convention, not a law of nature. Different fields use different standards depending on how much certainty they need. In genetics research, the threshold is often set at 0.00000001 or lower, because the sheer number of comparisons being made would generate too many false positives at 0.05. Some researchers have argued that even mainstream science should move to a stricter cutoff of 0.005 to reduce the number of false-positive findings that later fail to replicate.
This means a result with a p-value of 0.03 would be considered significant under the standard 0.05 threshold but not significant under the stricter 0.005 standard. The data are exactly the same; only the bar for declaring significance changes. That alone should tell you that “significant” and “not significant” are not clean categories carved into reality. They’re judgment calls based on agreed-upon conventions.
Statistical Significance vs. Practical Importance
Even when a result is statistically significant, it may not matter in any practical sense. And a result that’s not statistically significant might still point to something clinically meaningful. These two concepts, statistical significance and practical importance, are independent of each other.
Consider two cancer drugs tested in separate studies. Drug A increases patient survival by five years with a p-value of 0.01. Drug B increases survival by five months with the same p-value of 0.01. Both results are statistically significant, but the real-world difference between an extra five years and an extra five months is enormous. Statistical significance tells you only that the numbers are unlikely to be a fluke. It says nothing about whether the size of the effect matters to actual patients.
The reverse is also true. A study might find that a new rehabilitation program helps stroke patients regain meaningful function, but if the study was too small, the p-value could land at 0.08, just above the cutoff. Labeling that result “not significant” and dismissing the program would ignore potentially important clinical information. This is why the American Statistical Association issued a formal statement emphasizing that scientific conclusions should not be based only on whether a p-value passes a specific threshold, and that a p-value does not measure the size of an effect or the importance of a result.
How to Read Non-Significant Results
When you encounter a study described as having “no significant difference,” resist the urge to translate that as “no difference.” Instead, ask a few questions. How large was the study? A trial with 50 people has far less authority to rule out an effect than one with 5,000. What was the actual p-value? A result at p = 0.06 tells a very different story than one at p = 0.85. And what did the confidence interval look like?
A confidence interval gives you a range of plausible values for the true effect. If a study finds no significant difference but the confidence interval is extremely wide, stretching from a large benefit to a large harm, the study simply didn’t have enough data to say anything definitive in either direction. On the other hand, if the confidence interval is narrow and clustered tightly around zero, that’s much stronger evidence that any real effect is probably very small. The confidence interval is often more informative than the p-value alone.
In scientific papers, non-significant results are typically reported with phrases like “the difference was not significant” followed by the test statistic and a notation like “p = n.s.” (not significant) or the exact p-value. When you see this language, it’s a statistical claim about the evidence, not a factual claim about the world. The distinction matters every time you read a health headline that says a supplement “doesn’t work” or a risk factor “has no effect.” What the study likely showed is that it couldn’t detect an effect with the data it had, which is a much more cautious and honest statement.