The Will Rogers effect is a statistical paradox where moving individuals from one group to another raises the average in both groups, even though no individual’s outcome has changed. It’s named after American humorist Will Rogers, who joked during the Great Depression: “When the Okies left Oklahoma and moved to California, they raised the average intelligence level in both states.” In 1985, a physician named Alvan Feinstein borrowed this joke to describe something he kept seeing in cancer data: survival rates that appeared to improve in every stage of disease without any actual improvement in patient outcomes.
How the Paradox Works
The logic is surprisingly simple once you see the numbers. Imagine you have two risk groups, low and high, and three types of patients: those with good survival (80%), moderate survival (50%), and poor survival (10%). If the good and moderate patients land in the low-risk group while the poor-survival patients go to the high-risk group, the low-risk average is about 70% and the high-risk average is about 20%. The overall survival rate across everyone is 50%.
Now suppose a better diagnostic test reveals that the moderate-survival patients actually have features of high-risk disease. You move them into the high-risk group. The low-risk group, now containing only the 80% survivors, jumps to about 80%. And the high-risk group, which gained the 50% survivors to mix with its 10% survivors, rises to about 30%. Both group averages went up. But the overall survival rate is still exactly 50%, and not a single patient got healthier. The “improvement” is entirely an artifact of reclassification.
Why It Keeps Showing Up in Cancer
The Will Rogers effect is most commonly discussed in oncology because cancer treatment decisions revolve around staging. Staging determines how far a cancer has spread, and it directly shapes which treatments a patient receives and what survival statistics they’re quoted. When new imaging technology comes along (CT scans replacing X-rays, PET scans replacing CT scans), doctors can detect tiny metastases that would previously have been invisible. Patients who looked like they had localized, early-stage cancer under older technology get reclassified as having more advanced disease under newer technology.
This creates stage migration: patients shift from a lower stage into a higher stage. The patients who migrate are, by definition, the worst performers in the lower stage (they had hidden spread all along) and the best performers in the higher stage (their spread is minimal enough that older tools missed it). Remove the worst from one group and add the best to another, and both groups look better on paper.
A study of gastric cancer comparing surgical staging approaches found exactly this pattern. Stage migration inflated five-year survival rates by 1% in the earliest stage, 2% in the next, 7% in the intermediate stage, and a full 15% in more advanced stages. The researchers concluded that this explained, at least partially, why certain surgical techniques appeared superior in survival comparisons, without any real survival benefit for individual patients.
The Effect Beyond Cancer
The Will Rogers effect isn’t limited to oncology. Any field where diagnostic criteria change can trigger the same paradox. A clear example comes from multiple sclerosis. Researchers studied 309 patients who had experienced a first neurological episode suggestive of MS. After one year, they classified each patient using two different diagnostic systems: the older Poser criteria and the newer McDonald criteria, which are more sensitive and diagnose MS earlier.
Under the older criteria, only 16% of patients were classified as having MS at one year. Under the newer criteria, 44% were. That’s a huge migration of patients from the “isolated episode” category into the “MS” category. The result was predictable. Among those still classified as having an isolated episode, the probability of reaching significant disability dropped from 11% to 7% when using the newer criteria. Among those classified as having MS, the probability dropped from 46% to 27%. Both groups looked like they had a better prognosis, but the patients themselves were unchanged. The newer criteria simply moved moderate-severity patients out of the milder group and into the MS group, pulling both averages in a favorable direction.
Why This Matters for Medical Evidence
The practical danger of the Will Rogers effect is that it can make it look like medicine is getting better at treating a disease when really it’s just getting better at categorizing patients. If a hospital adopts newer imaging and then compares its stage-specific survival rates to historical data from five years earlier, the improvement could be entirely artifactual. No treatment changed. No patient lived longer. The numbers just shifted.
This is a particular problem for clinical trials that rely on historical controls, where a new treatment’s results are compared to previously published survival rates rather than to a concurrent control group. If diagnostic criteria changed between the historical period and the current study, the comparison is contaminated. The 2008 MS study made this point explicitly: using historical controls for MS treatment trials could generate spurious evidence of benefit.
Researchers have developed statistical methods to correct for stage migration, typically involving transition matrices that account for the probability that a patient would be classified differently under old versus new criteria. But these corrections require knowing the relationship between the two classification systems, which isn’t always straightforward. The simpler safeguard is to always look at overall survival across all stages combined, not just stage-specific rates. If overall survival hasn’t budged while every individual stage looks better, the Will Rogers effect is the likely explanation.
Recognizing the Pattern
The Will Rogers effect belongs to a family of statistical illusions where the composition of groups changes without the underlying reality changing. It’s related to Simpson’s paradox, where trends that appear in subgroups reverse or disappear when the data is combined. The distinguishing feature of the Will Rogers effect is that reclassification makes every subgroup appear to improve simultaneously, which is an especially convincing illusion because it looks like universal progress.
You can spot the conditions for it whenever three things are true: a population is divided into categories, the boundaries of those categories shift (through new technology, new diagnostic criteria, or new definitions), and individuals near the boundary tend to be the best performers in the more severe category and the worst performers in the less severe one. Those three ingredients reliably produce the paradox, whether you’re looking at cancer staging, neurological diagnoses, school district rankings, or any other system where people are sorted into tiers.