A one‑sample test is not a formula to memorise; it is a procedure, and it is the same procedure every time. State the hypotheses. Fix the error rate you will tolerate and check that the model is allowed. Standardise the result. Find the tail area. Decide, and say what it means in the sentence of the problem. Change the parameter, change the units, change the direction of the claim — the five steps do not move. Below you can pick a study, set the world it comes from, actually draw a sample, and walk the same five steps over it while every number recomputes. Then draw a different sample and watch the arithmetic change while the procedure does not.
Step 1 · State the hypotheses
The sample you actually drew
These are real observations, generated one at a time from the world set by the sliders on the right. Nothing here is a stored answer: press Draw another sample and the sample statistic, the standard error, the test statistic, the p‑value and the decision are all recomputed from the new numbers. The five steps never change — only what they are fed.
Sample size n—
Sample statistic—
Spread used—
Std. error of the estimate—
Run the whole procedure again and again
One sample gives one decision, and a single decision can be wrong. Here the five steps are carried out from scratch on fresh samples drawn from the same world — each square is one complete test, red if it rejected H0. This is what it means to say the procedure has a long‑run error rate: the rate belongs to the method, not to any one study, and not to the hypothesis.
Tests run0
Rejected H00
Rejection rate—
Should be near—
Three studiespick one and walk it through
The question on the table
The study, and the world it came from
The top slider sets what is actually true in the population — something a real analyst never gets to see, and something none of the five steps is ever allowed to use. The other controls are the two choices that are genuinely yours: how much data to collect, and how much false‑alarm risk to accept.
—
—
significance level α — fixed before the data arrives
Sample statistic—
Standard error—
Test statistic—
p‑value—
The five steps
The write‑up, as it would be handed in
All five steps at once, with this sample’s numbers in them. This is the shape of the answer for any one‑sample test — swap the parameter and the model and the five lines survive untouched.
Where the wording goes wrong
The order is not decorative. Steps 1 and 2 must be finished before the data is looked at, because both of them are choices that the data could otherwise be allowed to make for you. If you pick the direction of Ha after seeing which way the sample fell, you get to test whichever tail happens to look good — and the true false‑alarm rate of that procedure is roughly double the α printed next to it. If you pick α after seeing the p‑value, α has stopped being a risk you accepted and become a description of a number you already have. Steps 3 and 4 are pure arithmetic with no judgement in them at all. Step 5 is where judgement returns, and where the sentence has to be about the population, in its own units, with the direction and the level named.
“Fail to reject” is not “accept.” A large p‑value means the data would be unremarkable if H0 held. It does not mean H0 holds, because the same data would usually also be unremarkable under a great many nearby alternatives. Set n to its smallest value on any study here, push the truth well away from the null value, and watch test after test fail to reject something that is plainly false: the procedure did not find evidence, which is a statement about the instrument, not a verdict on the world. A test can accumulate evidence against H0 and never for it.
The p‑value is not the probability that H0 is true. Every number in step 4 is computed inside the world H0 describes; the calculation never asks how plausible that world was to begin with, and nothing in the five steps supplies that information. p is P(a result at least this extreme | H0 true), not P(H0 true | this result). In the same way, α is not the chance you are wrong on this study — it is the share of studies that would reject in the long run if H0 were true, which is exactly what the grid of repeated tests measures when the truth slider is parked on the null value.
One last habit worth building: after step 5, ask the question the test never asked. A rejection says a difference is more than noise; it does not say the difference is large enough to act on. Rejecting H0 for a bottling line running 0.3 mL off target is honest statistics and terrible advice. Report the estimate and an interval alongside the decision, and let the size of the effect argue for itself.