What Is Medical Evidence and Why Does It Matter?

Medical evidence is what careful research shows about whether a test or treatment actually helps people, as opposed to what seems to work or what has always been done. Research alone doesn’t settle good care, though. It’s meant to be weighed alongside a doctor’s judgment and what matters to you.1Himmelfarb Health Sciences Library Research Guides. Research Guides: Evidence-Based Medicine: What is Evidence-Based Practice? And because plenty of confident medical claims, from vitamins to common procedures, have later been reversed by better studies, knowing how evidence is judged helps you ask better questions.

The Three Ingredients of an Evidence-Based Decision

Doctors describe evidence-based medicine as the careful, open use of the best current research when deciding on care for one particular person.2PubMed. Evidence-based medicine That last phrase matters because research describes groups of people, while a decision has to fit a single patient. A doctor’s experience is what helps match general findings to your diagnosis and your situation, and it includes paying attention to your circumstances, rights, and preferences.3PubMed. Evidence-based medicine

The method is usually summarized as research plus clinical expertise plus patient values.4Aesthetic Surgery Journal. A Brief History of Evidence-Based Medicine (EBM) and the Contributions of Dr David Sackett In an appointment, research tells your doctor how a treatment did on average in people like you. Expertise covers spotting what makes your case different, such as other conditions or medicines you take. Your values cover trade-offs only you can rank, like side effects versus convenience. That’s why two people with the same diagnosis can reasonably choose different treatments. Tools called decision aids exist to lay out the options and help patients clarify their own goals.5Agency for Healthcare Research and Quality. Strategy 6I: Shared Decisionmaking

The ingredients can pull in different directions. Research may favor one option on average while your doctor’s experience suggests it’s a poor fit for you, for instance because you take another medicine that interacts with it. A good doctor then explains the mismatch and the reasoning, rather than quietly choosing for you or reciting the study’s result. Your values can pull against the research too: a treatment with a slightly better average result may not be worth it if it means daily injections or frequent trips to a clinic. It’s fair to ask which ingredient is driving a recommendation: the studies, the doctor’s experience with similar patients, or your stated preferences.

Why Some Kinds of Evidence Count More Than Others

Researchers rank evidence from most to least reliable:

Random assignment makes the groups alike except for the treatment, so differences in outcome point to the treatment. Observing people can mislead because those who choose a treatment may differ in health, income, or habits, and those differences, not the treatment, can explain better outcomes. Still, randomized trials can be flawed in ways that make their results questionable, and studies that observe people without assigning treatment shouldn’t be dismissed just for that reason.11PubMed Central. OBSERVATIONAL STUDIES VS. RANDOMIZED CONTROLLED TRIALS: AVENUES TO CAUSAL INFERENCE IN NEPHROLOGY

A review of many studies beats any single study for a few reasons. One trial can look good or bad partly by chance, especially if it’s small. A systematic review searches for all the studies on a question using a set method, rather than picking the ones that suit an argument, and then weighs them together. Pooling smooths out the luck of any one trial and shows whether the studies tend to point the same way. But a review is only as good as what goes into it: combining weak or biased studies gives a tidy-looking answer that is still unreliable.

Animal and lab studies sit even further from your care. Only about 12% of experimental drugs pass lab and animal testing to reach human trials, and about 89% of new drugs tested in people still fail.12PubMed Central. Limitations of Animal Studies for Predicting Toxicity in Clinical Trials Roughly half of those failures come from harmful effects in humans that animal tests didn’t predict.13PubMed Central. Limitations of Animal Studies for Predicting Toxicity in Clinical Trials A headline about a promising result in mice is a very early step. To place a news story on the ladder, check whether the study was in people, animals, or cells, and whether people were randomly assigned or just observed.

How Bias Can Tilt the Results You Hear About

Not everything that’s studied gets reported. Results showing a clear effect are about two to nearly five times more likely to be fully reported than results showing none.14PubMed Central. Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias The published record can therefore look more positive than the full body of research really was. A harm that appeared in unpublished studies may never come to light. Registering trials publicly before they begin lets others check whether the planned results ever appeared, which makes this kind of selective reporting easier to spot.

Who pays can shape what’s found, too. Dozens of studies link drug company funding of trials to results that favor the company.15PubMed. Evaluating solutions to sponsorship bias Company-sponsored studies were about four times as likely to favor the sponsor’s product as studies with other funding.16PubMed Central. Pharmaceutical industry sponsorship and research outcome and quality: systematic review In one analysis, trials run with the maker’s involvement reported the drug as 49% more effective than similar trials without it.17Drug Discovery News. Sponsorship effect: how drug manufacturers bias clinical trials Funding doesn’t prove a result wrong, but it’s a reason to look for independent confirmation.

You can check funding yourself. Most journal articles list funding and conflicts of interest near the end. News stories often omit this, so look up the original if you can. Independent or government-funded confirmation adds confidence.

Bias isn’t the same as fraud. Choosing comparison treatments or outcomes can tilt results without any lying, which is why the overall body of evidence matters more than one paper.

Many findings also don’t hold up when repeated. Large efforts to repeat biomedical experiments have had real trouble getting the same results,18PubMed. Reproducibility Failure in Biomedical Research: Problems and Solutions and some researchers argue that a large share of published findings may be false.19PubMed Central. Why Most Published Research Findings Are False A result confirmed by several independent teams is far more trustworthy than a single study.

When a Better Test Number Doesn’t Mean You’ll Live Longer

Some trials measure something like a blood value or tumor size in place of how patients feel, function, or survive.20PubMed Central. A framework for the definition and interpretation of the use of surrogate endpoints in interventional trials Trials use these stand-in measures because they can be smaller and shorter.21PubMed Central. Biomarkers and Surrogate Endpoints In Clinical Trials But smaller trials also give weaker reassurance about safety.22PubMed Central. Biomarkers and Surrogate Endpoints In Clinical Trials A marker can move in step with a disease without being part of what causes harm, which makes it misleading, and stand-ins can miss side effects that change whether a treatment truly helps.23PubMed Central. Biomarkers and Surrogate Endpoints In Clinical Trials

Blood pressure and cholesterol readings are familiar examples, since doctors generally care about heart attacks and strokes, not the number itself. For many treatments, improving the number does go along with fewer heart problems, which is why such markers are useful. The strongest proof is a trial showing the outcome itself changed, not only the marker.

Fast-track approvals rely on these measures. Accelerated approval in the US and conditional approval in Europe let drugs reach patients earlier based on stand-in measures.24PubMed. Era of surrogate endpoints and accelerated approvals: a comprehensive review on applicability, uncertainties, and challenges from regulatory, payer, and patient perspectives Of 55 cancer drug approvals based on stand-ins from 2009 to 2014, about two-thirds had no studies checking whether the measure tracked survival.25PubMed Central. Surrogate endpoints in oncology: when are they acceptable for regulatory and clinical decisions, and are they currently overused? Of those that were checked, only 3 closely tracked survival.26PubMed Central. Surrogate endpoints in oncology: when are they acceptable for regulatory and clinical decisions, and are they currently overused?

Earlier access can still be worth it. For serious diseases with few options, waiting years for full proof has a cost. The trade-off is more uncertainty about benefit and safety, and knowing a drug was fast-tracked lets you weigh that honestly.

With a new drug, ask whether it was shown to help people live longer or feel better, or only to change a test result. Ask how long it has been studied and in how many people, and whether follow-up studies confirming real benefit are under way.

How to Tell a Big Benefit From a Small One

Relative risk is the percent change in risk, such as “cuts heart attacks by half.” If your risk was 2 in 100, halving it to 1 in 100 changes your actual chances by just 1 in 100. That second figure is the absolute risk, the actual change in your chances.

The same percent change in risk also applies very differently to different starting points: halving a 20 in 100 risk helps ten times as many people as halving a 2 in 100 risk. That’s why a treatment can be well worth it for one person and barely matter for another.

The framing changes choices. In one study, about 57% of patients preferred a drug described by percent reduction, versus about 15% for the same benefit stated as an actual change in chances.27PubMed. The framing effect of relative and absolute risk Stating the actual change in chances helps people understand benefits most accurately and sways them less toward treatment.28PubMed. Evidence-based risk communication: a systematic review So when you hear a percentage, ask what your starting risk is.29PubMed. Expressing medicine side effects: assessing the effectiveness of absolute risk, relative risk, and number needed to harm, and the provision of baseline risk information

To work through a headline, find the starting risk without treatment and the risk with treatment. The gap between them is what you would actually gain, and you can compare that against side effects and cost. With your doctor, ask: out of 100 people like me, how many are helped, and how many get side effects?

“Statistically significant” is a separate trap. It means the difference is unlikely to be chance, not that patients would notice it.30PubMed. Research Pearls: The Significance of Statistics and Perils of Pooling. Part 1: Clinical Versus Statistical Significance A treatment can pass that test while changing symptoms too little for people to feel.31PubMed. Research Pearls: The Significance of Statistics and Perils of Pooling. Part 1: Clinical Versus Statistical Significance Researchers set separate thresholds for the smallest change that matters to patients,32PubMed. The difference between statistical significance and clinical relevance. The case of minimal important change, non-inferiority trials, and smallest worthwhile effect so it’s worth asking whether the improvement is one people usually notice.

Why Medical Advice Changes Over Time

A review of 146 established practices that were later contradicted found reversal is common across all areas of medicine.33PubMed. A decade of reversal: an analysis of 146 contradicted medical practices Beta-carotene became popular for cancer prevention in the early 1980s, but three randomized trials overturned the claim by the mid-1990s.34PubMed Central. Medical Reversal: Why We Must Raise the Bar Before Adopting New Technologies Even so, nearly a decade passed before arguments for it became rare,35PubMed Central. Medical Reversal: Why We Must Raise the Bar Before Adopting New Technologies and some procedures stay in routine use years after strong trials challenged them.36PubMed Central. Medical Reversal: Why We Must Raise the Bar Before Adopting New Technologies If your doctor changes advice after new research, that is the system working.

A wrong practice can become standard for understandable reasons. Many early practices rest on reasonable biology, such as a vitamin that seems to protect cells in lab studies, so it seems sensible to assume it will help people. Observational studies can also show that people who take something are healthier, even if the thing itself is not the cause. Only a large trial that tests the idea directly can show whether it works, and some ideas become widely used before that test is done.

A reversal isn’t proof medicine is unreliable. Changing course is how evidence corrects earlier mistakes, and advice built on stronger studies is less likely to be reversed. If advice you followed is reversed, don’t stop a medicine abruptly without talking to your doctor. Ask what new evidence prompted the change and whether it applies to people in your situation.

Recommendations are also graded. Guideline panels rate both how good the evidence is and how strong the recommendation should be.37PubMed Central. Grading quality of evidence and strength of recommendations Strength depends on balancing benefits and harms, confidence in the evidence, patient values, and cost.38PubMed. GRADE guidelines: 15. Going from evidence to recommendation-determinants of a recommendation’s direction and strength A “weak” recommendation means the choice depends more on your preferences.

Does the Evidence Apply to Someone Like You

When groups of people are underrepresented in trials, no one can be sure a treatment works the same way for them.39Open Access Journal of Clinical Trials. Bridging the Generalizability Gap: A Methodologic Framework for Enhancing Generalizability in Clinical Trials Across Underrepresented Populations It’s reasonable to ask whether people of your age, sex, background, or health conditions were included. Trials often exclude older adults, people with several illnesses, or those taking many medicines, to keep results easier to interpret. Yet those are often the very people who end up using the drug once it is approved. If you’re in a group that was left out, the results may still hold, but there is less direct evidence, so your doctor may want to monitor you more closely.

Tightly controlled trials can differ from your life in other ways. Participants are often healthier and take fewer other medicines, and they’re monitored closely and reminded to take doses. Everyday results can therefore be smaller, or side effects more common. Useful questions: who was studied and for how long, whether there is real-world data since approval, and whether your other conditions change the balance of benefit and harm.

Real-world studies help fill the gaps. Some trials test treatments in everyday care to see how well and how safely they work in real life.40PubMed. Pragmatic trials and real world evidence: Paper 7. Safety, quality and monitoring They aim to combine the rigor of randomized trials with the realism of observing ordinary practice.41Cardiovascular Prevention and Pharmacotherapy. Pragmatic Clinical Trials for Real-World Evidence: Concept and Implementation The FDA also runs a system called Sentinel that tracks drug safety using data from many health records sources.42PubMed Central. A Review of the Importance and Relevance of Real-World Data and Real-World Evidence