What is sampling bias?
Sampling bias is when the people or things you counted aren’t like the whole group you want to know about, so your answer comes out wrong in a predictable direction. Poll only the people who pick up a landline at 2pm on a weekday and you mostly hear from retirees. The fix isn’t asking more of the same people. It’s asking the right mix, which usually means picking at random.
How it happens
Most samples are convenient: whoever was easy to reach, whoever walked by, whoever bothered to answer. Convenient groups tend to have something in common, and that something leaks into the result.
Asking more people doesn’t fix it. Two hundred people who all share the same slant give you the same slanted answer, just with more confidence.
The common kinds
Convenience sampling
Asking whoever is handy. A survey of the people at the gym at 6am says a lot about morning people and not much about anyone else.
Self-selection
Letting people choose whether to answer. Online reviews and call-in polls hear mostly from people who feel strongly, which is why so many ratings are either five stars or one.
Survivorship bias
Counting only what made it through. Study just the businesses still open and every habit they share looks like the secret to success, because the ones with the same habits that closed aren’t in your data.
Undercoverage
Leaving a group out entirely. An online-only survey can’t hear from people who aren’t online.
The fix: a random sample
A random sample gives everyone in the group the same chance of being picked. That spreads the hidden differences out evenly, so the sample looks like the group. It’s why a carefully drawn random sample of about a thousand people can describe a whole country, while a million volunteer answers can still be wrong.
The course puts it in five words: twenty random beats two hundred convenient.
How to spot it
Ask two questions about any number you read. Who got counted? And who couldn’t have been? If the answer to the second one is a group that would have answered differently, be careful with the result.
Questions people ask
What is an example of sampling bias?
A restaurant asking diners at the end of their meal whether they liked it. The people who hated it walked out early or never came back, so the survey only hears from the ones who stayed.
Does a bigger sample fix sampling bias?
No. A bigger sample shrinks random error, but if everyone in it shares the same slant, you just get the same wrong answer with more confidence. Fix who you ask before you worry about how many.
What’s the difference between sampling bias and selection bias?
Sampling bias is one kind of selection bias. Selection bias is any way the data you end up with differs from the group you care about, whether it happened when you picked people, when they chose to answer, or when some dropped out.
Learn it by doing it
Module 6, “Good Questions, Fair Data,” covers this in its third lesson, “Who Got Counted?”, along with how to turn a hunch into a question you can actually answer.
Data to AI is five-minute lessons on how AI works, starting with the data on your phone, for curious teens, parents and grandparents. No math needed. The first lesson runs right here in your browser.