An unbiased sample is a subset of a larger group that accurately represents that group, giving every member an equal chance of being selected. If you wanted to know the average height of all adults in your city, you couldn’t measure everyone. So you’d pick a sample. That sample is unbiased when the people you pick reflect the full population, with no group over- or under-represented because of how you chose them.
What Makes a Sample Unbiased
The core idea is straightforward: if you could repeat your sampling process thousands of times, the average of all those samples would match the true value in the full population. A biased sample, by contrast, consistently skews in one direction. Polling only college students about national political opinions, for instance, would systematically miss the views of older adults, rural residents, and people without higher education.
Two conditions make an unbiased sample possible. First, you need a sampling frame, which is simply a complete list of everyone in the population you’re studying. Second, you need a random selection method that gives each person on that list a known, nonzero chance of being chosen. When both conditions are met, the sample’s characteristics tend to mirror the population’s characteristics, and the conclusions you draw are trustworthy.
Sampling Methods That Reduce Bias
There are four main approaches researchers use, each suited to different situations.
Simple random sampling is the most intuitive. You have your complete list of the population, and you select people at random, often using a computer-generated random list. It’s the gold standard when the population is small enough to list entirely.
Stratified random sampling divides the population into subgroups first, based on characteristics like age, gender, income level, or diagnosis. Researchers then randomly select people from each subgroup. This is especially useful when a minority group might be underrepresented in a purely random draw. If 5% of your population belongs to a particular ethnic group, simple random sampling might accidentally pull very few of them. Stratified sampling guarantees adequate representation from every subgroup.
Systematic sampling uses a fixed interval. You might select every 10th patient who visits a clinic, or every 20th name on a voter roll. You pick a random starting point and then follow the pattern. This works well when a full list isn’t available but people pass through a location or system regularly.
Cluster sampling is designed for very large populations where building a complete list of individuals would be impractical. Instead of listing every primary school student in a country, for example, you’d list all primary schools, randomly select some of those schools, and then randomly select students within them. It involves multiple rounds of random selection, each narrowing the group down.
Common Sources of Bias
Even well-designed studies can end up with biased samples. The most common culprits fall into a few categories.
Selection bias happens when the method of choosing participants favors certain types of people. Conducting a health survey only through a smartphone app excludes people without smartphones, who tend to be older or lower-income. The sample looks different from the population before a single question is asked.
Non-response bias creeps in when the people who agree to participate differ meaningfully from those who decline. A workplace satisfaction survey sent by email might get enthusiastic responses from employees who love their jobs and silence from those too disengaged to bother. The results look rosier than reality. Similarly, a survey sent during business hours may miss people who work night shifts, skewing the sample toward daytime workers.
Response bias occurs when participants answer untruthfully, often to align with what feels socially acceptable. People tend to underreport alcohol consumption, overreport exercise habits, and give answers they think the researcher wants to hear. The sample itself might be well-chosen, but the data it produces is still skewed.
Why Sample Size Matters
An unbiased selection method is necessary but not sufficient on its own. The sample also needs to be large enough to produce reliable results. There’s an inverse relationship between sample size and margin of error: the more people you include, the smaller your margin of error becomes. This is because the margin of error shrinks as you divide by a larger number in the underlying math.
In practical terms, a well-chosen sample of 100 people will give you a rougher estimate than a well-chosen sample of 1,000. Both can be unbiased, but the larger sample will produce results that land closer to the true population value more consistently. This is why major national polls typically survey at least 1,000 people, even when the population numbers in the hundreds of millions.
That said, a huge sample doesn’t fix a flawed selection method. A survey of 500,000 people drawn entirely from one social media platform is still biased, no matter how large it is. Size reduces random error. Proper sampling reduces systematic error. You need both.
How Researchers Correct for Bias
Perfectly unbiased samples are rare in practice, especially with the rise of online surveys. To compensate, researchers use a technique called weighting. The idea is simple: if your sample has too few young men compared to the actual population, you give each young man’s response more mathematical weight so the final results better reflect reality.
The weights are calculated by comparing the proportion of each demographic group in the sample to their proportion in the actual population. If women aged 18 to 29 make up 12% of the population but 20% of your survey respondents, their answers get scaled down. If older men are underrepresented, their answers get scaled up. Weighting typically adjusts for gender, age, ethnicity, education level, and geographic area.
Weighting helps, but it has limits. It can correct for known imbalances in measurable demographics. It can’t fix biases tied to unmeasured differences, like personality traits that make some people more willing to take surveys in the first place.
Spotting Bias in Polls and Studies
The American Association for Public Opinion Research (AAPOR) maintains transparency standards for survey reporting. When evaluating whether a poll’s sample is trustworthy, look for a few key disclosures: how the sample was constructed (probability-based or not), the sample size, the margin of sampling error, how participants were recruited, the survey mode (phone, online, in-person), and whether weighting was applied.
Probability-based samples, where every person in the population has a known chance of selection, have long been considered the gold standard. Nonprobability samples, like opt-in online panels where people volunteer to participate, are increasingly common but carry higher risk of bias because the people who sign up may differ systematically from those who don’t. When a poll doesn’t disclose its methodology, that alone is a reason to treat its findings with skepticism.