A post hoc analysis is any statistical analysis conducted after the original study has already been completed, looking at questions that weren’t part of the original plan. The term “post hoc” is Latin for “after this,” and it describes the timing perfectly: researchers collect their data to answer one question, then go back and ask additional questions of that same data. These analyses can reveal useful patterns, but they carry important limitations that affect how much weight you should give the results.
How Post Hoc Analysis Differs From Planned Analysis
Before a study begins, researchers write a protocol that spells out exactly what they’re testing and how they’ll measure it. This is called a prespecified or confirmatory analysis. It’s designed to answer one clear question with statistical rigor. A post hoc analysis, by contrast, happens after the data is already in hand. The researcher notices something interesting, or wants to explore a subgroup, or tests a relationship that wasn’t part of the original design.
This distinction matters enormously. In a confirmatory study, the hypothesis comes first and the data comes second. In a post hoc analysis, the data comes first and the hypothesis follows. A generated hypothesis cannot be confirmed using the same data that inspired it. It needs a separate, new study for that. Presenting a post hoc finding as though it were planned in advance is considered either methodological ignorance or scientific misconduct.
Why Post Hoc Results Are Considered Exploratory
The core issue is statistical. Every time you run an additional test on the same dataset, you increase your chances of finding a “significant” result purely by luck. With a standard significance threshold of 5%, one test gives you a 5% chance of a false positive. Run two tests and that risk climbs to roughly 9.8%. Three tests: 14.3%. By six tests, you have a 26.5% chance of finding at least one false positive. The more questions you ask of the same data, the more likely you are to get a misleading answer.
This is sometimes called alpha inflation, and it’s the mathematical reason post hoc findings are treated as hypothesis-generating rather than hypothesis-confirming. They can point researchers in a promising direction, but they don’t prove anything on their own.
Post Hoc Tests After ANOVA
There’s a second, more specific use of the term “post hoc” that comes up in statistics courses. When researchers compare three or more groups (say, three different treatments), they first run an overall test called ANOVA to see whether any differences exist at all. If that overall test is significant, meaning at least two groups differ, they then run post hoc comparison tests to figure out which specific groups differ from which. If the overall test isn’t significant, they stop there.
Several methods exist for these comparisons, and they differ mainly in how aggressively they guard against false positives:
- Tukey’s HSD: The most commonly recommended method when you want to compare every group to every other group. It provides straightforward control over false positives and works well with equal or unequal group sizes.
- Bonferroni correction: Divides the significance threshold by the number of comparisons. Simple to understand but becomes overly strict when you’re making many comparisons, which can cause you to miss real differences.
- Scheffé’s method: The most conservative option. It’s designed for complex comparisons (not just pairs) and has the tightest error control, but it’s more likely to miss genuine effects because of how strict it is.
- Dunnett’s test: Used when you only care about comparing each treatment group to a single control group, rather than comparing all groups to each other.
More liberal methods like Duncan’s multiple range test or the Student-Newman-Keuls procedure are easier to get significant results from, but they allow higher rates of false positives. The choice of method depends on whether you’d rather risk missing a real effect or risk claiming a false one.
Where Post Hoc Analysis Causes Problems
Post hoc analyses become problematic when they’re misused or misrepresented. Several well-known practices fall into this category. P-hacking involves relentlessly re-analyzing data with the goal of finding any statistically significant result. A fishing expedition means testing dozens of variable combinations without a specific question in mind, hoping something turns up. HARKing, or “hypothesizing after the results are known,” is when a researcher runs a post hoc analysis but writes up the paper as if they’d planned it all along.
These practices aren’t rare. They’re common enough that major reporting guidelines now require researchers to be transparent about them. The CONSORT 2025 guidelines, which are the international standard for reporting clinical trials, specifically require authors to distinguish prespecified analyses from post hoc ones. Researchers must report any outcomes or analyses that weren’t in the original plan and flag any changes made after the trial started.
When Post Hoc Analysis Is Genuinely Useful
Despite these limitations, post hoc analysis plays an important role in research, particularly in medicine. Clinical trials are designed around a primary question, but the data collected often contains clues about which patients benefit most, which might be at higher risk for side effects, or whether a treatment works differently in certain subgroups.
A well-known example comes from breast cancer research. A genetic test can identify women with early-stage hormone receptor-positive breast cancer who may not need chemotherapy. Post hoc subgroup analysis by age found that chemotherapy could be safely skipped for women over 50 with lower recurrence scores and for women 50 and under with even lower scores. That kind of finding can reshape treatment decisions for thousands of patients, but it started as an exploratory analysis that then needed confirmation in further studies.
Post hoc analyses can also flag safety signals that weren’t anticipated when the trial was designed. A drug might show unexpected side effects in a specific population, or a treatment might work for a condition it wasn’t originally being tested for. These discoveries don’t count as proof, but they tell researchers where to look next.
How to Evaluate Post Hoc Findings
If you’re reading a news article or study that reports a post hoc result, a few things are worth checking. First, does the paper clearly label the finding as post hoc or exploratory? Trustworthy research is upfront about this. Second, has the finding been replicated in a separate, prespecified study? If not, it’s a lead, not a conclusion. Third, how many comparisons were made? A single post hoc finding pulled from dozens of tested subgroups is far less reliable than one pulled from two or three.
The practical takeaway is straightforward: post hoc analyses generate questions, and confirmatory studies answer them. Both steps are necessary. A post hoc finding that gets confirmed in a new trial is just as valid as any other scientific result. One that hasn’t been confirmed yet is a promising clue that deserves appropriate skepticism.