Somebody makes a claim about how a whole population is divided up — a candy maker advertises a colour mix, a die is said to be fair, a theory predicts which genotypes should appear and in what ratio. You cannot check the claim by measuring an average, because there is no average to measure; the data are counts in categories. So you do the only sensible thing: you take the claim seriously for a moment, work out how many of each category it predicts in a sample your size, and compare those expected counts with the ones you actually got. Every category will be off by something — that is guaranteed, even when the claim is perfectly true. The question is whether the collection of misses is bigger than chance alone would produce. Below you set the claim and the hidden truth, open real bags, and watch a single number, χ², assemble itself one category at a time out of those misses.
One bag, counted colour by colour
The pale grey bars are the counts the advertised mix predicts for a bag this size: E = n × the claimed share. Nothing observed goes into them. The coloured bars are what this particular bag actually held. Press Open a new bag a few times and watch every colour miss its target every single time — even when the claim on the right is set to be exactly true. Misses are not evidence of anything by themselves. Their size is.
expected under the claimobserved in this bag
Building χ², one cell at a time
A raw miss of “7 too many” means something very different in a colour you expected 8 of than in one you expected 200 of. So each miss is squared, to kill the sign and punish big misses harder, and then divided by its own expected count, to grade it against how many you were waiting for. That gives one number per category. Add them up and you have χ². Tap any row to see its arithmetic in full.
Colour
Observed O
Expected E
O − E
(O−E)² ÷ E
How surprising is that total?
χ² can only be zero or positive, and it is zero only if every count lands exactly on its expectation. So the whole question lives in one tail: is this total too big to be ordinary noise? The curve below is the distribution χ² follows when the claim is true, with df = k − 1 degrees of freedom. The shaded region to the right of your total is the p‑value. The dashed line is the point past which we would reject at α = 0.05.
χ² if the claim were truethis big or bigger — the p‑valueyour totalreject beyond here
Open a thousand bags
One bag settles nothing. So repeat the entire study: each run below fills a fresh bag from the true mix, scores it against the claimed mix, and keeps only the χ². The bars are those simulated totals. The dark curve is the theoretical χ² distribution with df = k − 1 — drawn from the formula, fitted to nothing. Set the excess on the right to zero and the bars land on the curve; nudge it up and the whole pile marches right, which is exactly what statistical power looks like.
simulated bagstheory, not fittedrejection cut‑off
Bags opened0
Mean χ² observed—
Mean χ² predicted—
Share rejecting—
A claim about a mixset it up, then start opening bags
The claim, and the truth behind it
In a real study you would know only the claim; the truth is what you are trying to find out. Here you set both, so you can watch the machinery succeed and fail against an answer you already have.
What is being claimed
The study
6
200
The hidden truth
+8 pts
Jump to a situation worth seeing
This bag, computed live
Candies counted n—counts, never %
Categories k—colours claimed
Degrees of freedom—k − 1
Smallest expected E—
χ² statistic—Σ (O−E)²/E
Reject beyond—α = 0.05
p‑value upper tail—
Biggest contributor—
Is the curve even legal here?
The χ² curve is an approximation. The statistic itself is discrete — it can only take the values the counts allow — and the smooth curve only mimics it well once every expected count is reasonably large. The usual working rule is that every expected count should be at least about 5. Note carefully: the rule is about E, not O. Observing zero of a colour is perfectly fine; expecting half of one is not.
Smallest expected—
Cells with E < 5—
Smallest observed—
Bag size to clear the rule—
Say it right
Nine things people say after a goodness‑of‑fit test. Four of them are correct. Tap each one.
What to carry out of this section
Expected counts come from the claim, not from the data. E = n × the claimed share, worked out before you look at a single candy. That is what makes the comparison honest: you are not fitting anything, you are holding the data up against a prediction that was fixed in advance. If you ever find yourself computing expected counts from the observed ones, you have stopped testing the claim and started describing your sample.
Square it, then divide by what you were waiting for. The squaring removes the sign, so an excess and a shortfall are both just “wrong,” and it makes one large miss cost far more than several small ones. The division by E is the part people forget: it converts a raw miss into a miss relative to the scale you expected. Being 10 over on an expected 8 is enormous; being 10 over on an expected 400 is nothing. Only after that division are the categories on comparable footing and safe to add up.
df = k − 1, and the missing one is not arbitrary. The expected counts are forced to add up to n, the same total as the observed counts. So once you know how the first k − 1 categories missed, the last miss is fully determined — it has to close the books. Only k − 1 of the deviations are free to vary, and the curve you read the answer off knows it. Drag k and watch the whole curve change shape underneath the same data.
χ² scales with n, so never feed it percentages. Because O = n·ô and E = n·q, the statistic works out to n × Σ(ô − q)²/q. Hold the percentages fixed and multiply the bag size by ten and χ² multiplies by ten too. This is the single most common way the test is botched: a table of percentages typed in as though they were counts, producing a number with no meaning at all. Sample size is not a nuisance here; it is half the message.
Only the upper tail is used — and not because the alternative is one‑sided. Every kind of mismatch, in any category, in either direction, makes χ² larger. Too many reds and too few reds both inflate the same statistic, because the deviation is squared. So a big χ² is the only signal of misfit there is, and the lower tail is not a rejection region. What the lower tail is good for is suspicion: counts that hug their expectations far more closely than chance allows produce an oddly small χ², and that pattern has been used more than once to question whether a famous data set was really collected the way it was reported.
A significant χ² does not tell you which category is at fault. This is an omnibus test: it answers one question, “does the whole claimed mix fit?”, and it answers it with one number. The per‑cell contributions in the second panel are the natural place to look next, and the square root of a contribution, (O − E)/√E, is the standardised residual you would report. But hunting through k cells after the fact to find the guilty one is a fishing expedition, and any single cell picked that way deserves confirmation in fresh data rather than a p‑value borrowed from the omnibus test.
Failing to reject is not a verdict of “the claim is correct.” Load the Barely off, huge bag preset and then drag the bag size down to 100. The truth has not moved a millimetre — the mix is still genuinely wrong — but the test now shrugs at it, because a bag that small cannot resolve a gap that fine. A large p‑value means the data were compatible with the claim, and small data are compatible with almost everything. It is a statement about your resolution as much as about the world.
One last thing worth doing. Set the excess to zero, so the claim is exactly true, and run five thousand bags. The mean χ² settles on df = k − 1 — that is not a coincidence, it is a theorem — and about one bag in twenty crosses the rejection line anyway. Those bags contain nothing unusual. They are ordinary bags from an honest factory, and they are the advertised, agreed‑upon cost of drawing a line at 5%.