Innocent Until Proven Significant: A Gentle Intro to Null and Alternative Hypotheses
Statistics is a field I genuinely love. Unfortunately, the feeling isn’t always mutual. Every now and then, it acts like that one professor who says “this is trivial,” fills three blackboards with Greek letters, and turns around to find the entire class silently questioning their life choices.
Thankfully, there are people like Josh Starmer, whose YouTube videos make statistics far less scary (BAM!). But I’m the kind of person who asks “why?” and “how?” so often that even I find it annoying, and high-level videos weren’t enough to keep my inner toddler quiet.
So when I recently found some downtime, I decided to give statistics another shot, and this time do it properly. No more nodding along to videos. I went down the rabbit hole of articles and research papers, armed with coffee and a dangerous amount of curiosity.
Somewhere along the way, I ran into an old enemy: the null hypothesis. The first time I met it, I read the definition, understood nothing, and confidently rejected the null hypothesis, with zero evidence. (If you just laughed, you already know your stats. If you didn’t, perfect. By the end of this post, you will.)
So I did what any sore loser would do: I studied the enemy. I pulled up every reference I could find, connected the dots, and realized two things. First, the concept is actually pretty simple. Second, plenty of people have explained it well, but the pieces are scattered all over the internet, and most of them tell you what to do without ever explaining why.
Here it is: my attempt to make the null hypothesis (and its partner in crime, the alternative hypothesis) easier for you than it was for me. Why do we need them? What are they? How do they help? When do we use them? And, if you stick around, a sneak peek at the villain of my next post: the p-value.
It All Starts With a Sample
Before we meet the null hypothesis, we need to understand the problem it was invented to solve. And that problem starts with a sample. Most research, whether in statistics, psychology, or medicine, follows the same basic recipe: pick a group, measure something about them, and summarize what you found.
That group is called a sample, a smaller, manageable slice of the bigger group you actually care about. The bigger group is called the population.
Why Not Just Ask Everyone?
Fair question. Well, if you have infinite money, infinite time, and an unhealthy love for spreadsheets, be my guest. For the rest of us, it’s practically impossible. India alone has roughly 1.48 billion people. Imagine asking every single one of them whether their toothpaste has salt in it. Even the lady from that famous toothpaste commercial didn’t try that. She just asked a handful of people near the billing counter. (Congratulations to her, by the way. She was doing sampling.)
From 500 Engineers to All of India
Once you’ve collected data from your sample, you summarize it. That summary is called descriptive statistics: numbers that describe your sample, like an average, a percentage, or how spread out the values are. For example, let’s say 8% of the 500 engineers we surveyed quit their jobs to become farmers. That 8% is a descriptive statistic.
But here’s the thing: serious researchers don’t really care about those 500 engineers. (Sure, the internet is full of “I had coffee with 3 founders and here’s why the economy is doomed” posts. That’s not research. That’s just coffee.) The real goal is to take what we learn from the sample and use it to say something about the entire population. That leap from sample to population is called inference.
Inference is the process of using what you observed in a sample to draw conclusions about the whole population. Think of it as an educated guess, one backed by data instead of vibes. Going back to our engineers: if 8% of the 500 engineers we surveyed became farmers, we infer that roughly 8% of all engineers in India have made the same switch. We never spoke to every engineer in the country, but our sample lets us make a reasonable claim about all of them.
Our engineers are fictional, but this process isn’t. It’s exactly how India estimated that about 1 in 9 adults has diabetes: researchers in the ICMR-INDIAB study tested around 113,000 adults across the country and inferred the number for all of India.
So, in short: descriptive statistics tells you about your sample. Inference is how you go beyond it.
Read the Fine Print
There’s a bit of fine print: this only works if your sample actually looks like the population. That means two things. First, it needs to be big enough, because 3 people can’t speak for millions. Second, it needs to come from the right group. Survey 500 engineers who all live in Bengaluru, and you’ve learned a lot about Bengaluru engineers, but not much about the rest of India. (This is also why our coffee-with-3-founders friend fails twice: too few people, and founders aren’t the whole economy.)
Tick both boxes, and your sample will get you close to the truth. But close isn’t exact. Think of tasting soup: take a decent spoonful (big enough) from a well-stirred pot (right group), and your spoon will taste a lot like the whole pot. Just not exactly like it. Maybe a little extra salt landed in your spoon. Your sample works the same way. So is 8% the true number for all engineers in India? Probably not, but it shouldn’t be far off. And that small gap between “close” and “exact” has a name: sampling error.
Sampling Error: Nobody Messed Up, It’s Just Luck
First, a quick clarification: sampling error doesn’t mean someone messed up. (People do pick terrible samples in real life, like surveying people at a Starbucks to estimate India’s average income. But that’s a different problem.) In statistics, sampling error is simply the difference between what your sample says and what’s true for the whole population, caused purely by the luck of who ended up in your sample.
Let’s go back to our engineers. Say you survey a fresh batch of 500 engineers, just as carefully as before. This time, 7.6% have become farmers. (Remember, the first batch gave us 8%.) Try again with another 500: 8.3%. Same country, same careful method, different answers. Nobody made a mistake. Statistics just likes to keep you humble. You got a slightly different spoonful each time.
The good news: bigger samples shrink this wobble. Survey 5,000 engineers instead of 500, and your results will land much closer to each other, and to the true number. The bad news: it never fully disappears unless you survey everyone, and we’ve already established that you don’t love spreadsheets that much.
So Why Is This a Problem?
Now, remember our 8%? Imagine a study from a few years ago had found that only 6% of engineers had switched to farming. Put the two together, and you get a headline: “More engineers than ever are turning to farming! It was 6%, now it’s 8%.” (Somewhere, a parent who paid for four years of engineering just fainted. Me? I’m low-key jealous.)
Sounds dramatic. But you now know something the headline writer may have forgotten: every sample wobbles. So there are two ways to read this:
- Something real happened. More engineers really are becoming farmers, and our 8% reflects that.
- Nothing changed. Engineers are switching to farming at the same rate as before, and our sample just happened to catch a few extra farmer-engineers. Pure sampling error. (The headline writer would hate this option.)
Both stories explain the same jump from 6% to 8%. It’s like a courtroom where two lawyers point at the same evidence and tell completely different stories. And just by looking at the numbers, you can’t tell who’s right.
And this isn’t just about engineers. Any time you see a difference in a sample, whether it’s a new app feature that seems to boost sign-ups, a medicine that seems to work better, or students who seem to score higher after a new teaching method, the same two explanations are on the table: a real effect, or the luck of the draw. Statisticians have fancy names for these two stories, and you’re about to meet them.
The Null Hypothesis: The “Nothing Changed” Story
Meet the null hypothesis: statistics’ way of settling the “real or luck?” debate fairly, the same way every time. Remember story #2, the one where nothing changed and our 8% was just sampling error? That’s the null hypothesis.
Let’s break down the name. A hypothesis is simply a claim you can test, an educated guess like “more engineers are becoming farmers” or “this new feature boosts sign-ups.”
Now add “null,” which means zero, nothing, nada. So the null hypothesis (written as H₀) is the claim that nothing interesting is going on: no change, no difference, no effect. Whatever pattern you spotted in your sample is just sampling error playing tricks on you. Think of it as statistics’ resident party pooper: “Relax, it’s probably nothing.”
In our example, the old study said 6% and our sample says 8%. The null hypothesis says: “The share of engineers becoming farmers hasn’t changed. That jump? Just the luck of the draw.”
Why Start With the Boring Story?
You might wonder why statistics starts with the party pooper instead of the exciting claim. Two reasons:
- It’s specific. “Nothing changed” points to one clear number: 6%. That gives us something concrete to test against. “Something changed” could mean 1%, 4%, 8%, 15%, or literally any number that isn’t 6%, which is much harder to pin down.
- It keeps us honest. Humans love finding patterns, even in random noise. Starting from “it’s probably nothing” forces us to show real evidence before getting excited, like that one uncle in the family WhatsApp group who doesn’t believe a single forward without a source. (Okay, that uncle doesn’t exist. But he should.)
Innocent Until Proven Significant
And here’s where the courtroom comes in. We start by assuming the null hypothesis is true. Innocent until proven guilty. Our job isn’t to prove it right. It’s to check whether the evidence is strong enough to say otherwise.
Notice something sneaky here: the trial only ever examines the null. We never try to prove directly that something changed. We just ask one question: “If nothing changed, how likely is it that we’d see evidence like this?”
If you have a sibling, you’ve already run this test. You just didn’t know it.
The last slice of cake, the one you’d been saving all day, goes missing from the fridge. Your sibling swears they didn’t touch it. Let’s assume they’re innocent. That’s our null hypothesis. (Unlike most parents, statistics actually presumes innocence.) But then the evidence rolls in: frosting on their fingers, a trail of crumbs leading to their room, an empty plate hidden under their bed, and they’ve been suspiciously quiet for twenty minutes.
Now ask the question: if they really are innocent, how likely is it that all of this would show up together? Frosting on their fingers alone could be anything. Maybe they had a cupcake at school or licked a spoon while baking. But the crumbs, the plate, the silence, all at once? Very unlikely. Unlikely enough that “they’re innocent” stops sounding believable.
That’s exactly what we do with our engineers. If nothing changed and the true rate is still 6%, how likely is it that a sample would show 8% just by luck?
The Verdict: Reject or Fail to Reject
A trial can end in only two ways.
1. We reject the null hypothesis.
The evidence is so unlikely that “it’s just luck” no longer sounds believable. This is the full crime scene: frosting, crumbs, plate, silence. For our engineers, it would mean that if nothing had changed, getting 8% would be really unlikely, so something real probably happened. Note the word probably: rejecting the null isn’t absolute proof. It means the evidence is strong enough that we’re willing to bet against luck.
Get Soumabrata Sarkar’s stories in your inbox
Join Medium for free to get updates from this writer.
2. We fail to reject the null hypothesis.
The evidence isn’t unlikely enough. This is just frosting on the fingers, which could easily have an innocent explanation. For our engineers, it would mean a jump from 6% to 8% could reasonably happen by chance, so we can’t rule out luck. The null survives, for now.
Why We Never “Accept” the Null
Here’s the part that trips everyone up. If the null survives, why not just say we accept it?
Because a court never declares anyone “innocent.” It declares them “not guilty.” That difference matters. “Not guilty” means there wasn’t enough evidence to convict, not that the person definitely didn’t do it. (Your sibling knows this. That smug look when your parents say “fine, I can’t prove it” is not the look of an innocent person.)
Same with the null. Failing to reject it doesn’t prove nothing changed. Maybe something did change, but our sample was too small to catch it. It’s like losing your keys, checking one room, and declaring “my keys don’t exist.” You didn’t prove they’re gone. You just didn’t look hard enough.
Absence of evidence isn’t evidence of absence. So statisticians stay humble and say “we fail to reject the null,” which is a fancy way of saying “we couldn’t prove anything happened.”
But wait: if the null gets rejected, what exactly are we left believing? That’s the other story, and it has a name too.
The Alternative Hypothesis: The “Something Changed” Story
Remember story #1, where more engineers really are becoming farmers? That’s the alternative hypothesis, written as H₁ (or sometimes Hₐ). It’s usually the claim the researcher actually cares about, the one that would make the headline.
If the null is your sibling saying “I didn’t eat the cake,” the alternative is you pointing at the empty plate.
Two Stories, Written Together
H₀ and H₁ are a pair, so you write them down together, before you look at any data. They’re usually written so that, between them, they cover every possible outcome. Whatever the truth turns out to be, it falls under one of the two stories.
The simplest pair looks like this: if H₀ says “the rate is 6%,” then H₁ says “the rate is not 6%.” Between them, every possible number is covered.
Three Flavors of “Something Changed”
“Something changed” sounds simple, but you have to decide upfront what kind of change counts. And whichever you pick, the null quietly adjusts so the two stories still cover everything:
1. Changed in either direction. H₁: the rate is not 6%. H₀: the rate is 6%.
2. Increased. H₁: the rate is higher than 6%. H₀: the rate is 6% or lower.
3. Decreased. H₁: the rate is lower than 6%. H₀: the rate is 6% or higher.
Notice the null is still the party pooper in every case. It just means “nothing happened in the direction we care about.”
Our headline claimed engineers are turning to farming “more than ever,” so that’s flavor #2.
Why does this matter? Because it decides which evidence counts. If you bet on “it increased” and your sample comes back at 4%, that lands squarely in the null’s territory (“6% or lower”), and you can’t suddenly switch to “well, it changed!” That’s like calling heads, seeing tails, and announcing you meant tails all along. The rule is simple: pick your alternative before you peek at the data.
Two Stories, But Not Equal Treatment
Here’s the twist: the two stories aren’t treated equally. If the evidence is strong enough to reject the null, we side with the alternative. But if it isn’t, we don’t side with the null. We just say “not enough evidence,” the same “not guilty, not innocent” logic from before.
The alternative has to earn its win. The null just has to survive!
So We Failed to Reject. Now What?
This is the question that bugged me the most. We wrote two stories, ran the trial, and the null survived. But it didn’t win either. So… who’s right?
Honest answer: we don’t know yet. Both stories are still standing.
Back to the cake. Say the only clue you found was a little frosting on your sibling’s finger. That fits “innocent” (they licked the spoon while someone was baking) and “guilty” (they ate the cake and wiped the plate clean) equally well. You can’t convict. But you also can’t declare them innocent and apologize for accusing them. The case just stays open.
So what do you do with an open case?
- Collect more evidence. Check the dustbin. Question the other sibling. In statistics, that means a bigger sample. Bigger samples shrink the wobble, so if a real change exists, it becomes easier to spot. Survey 5,000 engineers instead of 500, and if the jump from 6% to 8% is real, luck will have a much harder time explaining it.
- Ask whether you ever stood a chance. Remember, small samples wobble a lot. Survey only 50 engineers, and 6% means 3 people while 8% means 4 people. The entire “jump” comes down to one extra farmer. That could easily be luck, so the test was never going to call it real, even if the change actually happened. In other words, a tiny sample can hide a real change inside its wobble. Failing to reject in that situation tells you almost nothing. It’s like searching for your keys with the lights off and concluding they don’t exist. (Statisticians call a test’s ability to catch a real change its “power,” if you ever want to look it up.)
- Make a practical call. Sometimes “we couldn’t detect a change” is enough to move on. If a new app feature showed no clear effect on sign-ups, a product team might shelve it. Just remember to say “no clear effect,” not “no effect.”
What you should not do: keep re-running tests on the same data until something comes out significant. That’s like asking your sibling the same question 47 times until they finally say “fine, I ate it.” You got an answer, just not one you can trust. (This has a name too, p-hacking, and yes, it’s coming up in a future post.)
And If We Do Reject?
Then the alternative wins, with an asterisk. We say something probably changed, not that it’s proven. A good researcher’s next questions are “by how much?” and “does it show up again in a fresh sample?” One trial gives you a verdict. Getting the same verdict a second time is much harder to argue with.
The Whole Trial in One Place
Verdicts, open cases, asterisks… that’s a lot of courtroom drama for one missing slice of cake. So let’s zoom out and put the whole trial on a single page, using our engineers:
Five steps. Two stories, one assumption, one question, and a verdict that either knocks down the boring story or sends you back to collect more evidence.
And this same courtroom runs everywhere: a shopping site testing whether a new “Buy Now” button really boosts sales, a drug trial checking whether patients really recover faster, or a cricket fan wondering whether a batter’s home average is real or just a few lucky innings. Different worlds, same trial.
The One Piece We Haven’t Measured
Look at the flowchart again, though. There’s one step we’ve been a little vague about: step 4. We keep saying “very unlikely” and “not unlikely enough,” but where exactly is the line?
Picture a football match between two sections of the same class, teams that are supposed to be evenly matched. If Section A wins 3-2, nobody blinks. Close games happen. If Section A wins 10-0, you start wondering whether someone quietly stacked their team with the school’s best players.
But what about 5-2? Or 6-1? Where does “probably luck” end and “something’s off” begin?
Statistics has a number for exactly that, called the p-value. Here’s the sneak peek.
The p-value answers the same question we’ve been asking all along: if the boring story (H₀) is true, how likely is it that we’d see a result like ours, or something even more extreme, just by luck?
In football terms: if the two sections really are evenly matched, how often would a game end 10-0, or something even more lopsided? Almost never. That’s a tiny p-value, and luck becomes hard to believe. A 3-2 result? That happens all the time between equal teams, so the p-value is large, and luck is a perfectly good explanation.
Before You Go
If you remember only one thing from this post, make it this:
Statistics starts by assuming nothing happened, and only changes its mind when the evidence makes “it was just luck” hard to believe.
Okay, that’s enough statistics for one day. Read it, let it sink in, and if anything still feels fuzzy, drop your doubts in the comments or reach out. I’ll be more than happy to help (asking “why?” is my favorite hobby, remember?).
See you in the next one!
References & Further Reading
- Josh Starmer (StatQuest), “Hypothesis Testing and the Null Hypothesis, Clearly Explained!!!” The video that made statistics feel less scary for me. BAM!
youtube.com/watch?v=0oc49DyA3hU - Price, Jhangiani & Chiang, Research Methods in Psychology (2nd Canadian Edition), “Understanding Null Hypothesis Testing.” A free, open textbook with a clear walkthrough of the null hypothesis testing logic.
opentextbc.ca/researchmethods/chapter/understanding-null-hypothesis-testing - Anjana et al. (2023), ICMR-INDIAB study, The Lancet Diabetes & Endocrinology. The real-world study behind the “1 in 9 adults” inference example.
doi.org/10.1016/S2213–8587(23)00119–5 - “A logical analysis of null hypothesis significance testing using popular terminology,” BMC Medical Research Methodology (2022). For the curious: how textbooks differ in writing H₀ and H₁.
bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874–022–01696–5 - Wasserstein & Lazar (2016), “The ASA’s Statement on p-Values: Context, Process, and Purpose,” The American Statistician. Homework for the next post, straight from the American Statistical Association.
doi.org/10.1080/00031305.2016.1154108
