Sampling and Inference

Updated 5 October 2026

Inference uses a sample to make a claim about a whole population. Every method below answers one question: how far can a sample be trusted to speak for the population it came from?

What is a sample, and what makes a good one?

A population is the whole group you want to know about, and a sample is the part you actually measure. A number describing the population, such as its mean, is a parameter; the same number computed from the sample is a statistic, and it estimates the parameter. See Populations and Samples. A good sample is chosen at random, so every member has a known chance of selection and the method favors no group. To survey 50 of a school's 1,000 students, let a random number generator pick 50 of them, a sampling fraction of $\frac{1}{20}$1/20, since 1,000 ÷ 50 = 20. Randomness removes bias: volunteers, or people easy to reach, differ from the rest in ways a larger sample cannot fix. When the population has groups of known size, a stratified sample takes from each in proportion: if 400 students are seniors, $400\times \frac{1}{20}=20$400 × 1/20 = 20 of the 50 should be seniors. Check that the strata add up: 20 + 30 = 50.

In a simple random sample every member has the same probability. In a systematic sample you take every tenth name on a list, which relies on the order of the list being fair. See Sampling Methods.

In an experiment the researcher assigns the treatment, for example by a coin toss. In an observational study people put themselves into groups, and those groups can differ before the treatment starts. Only random assignment supports a claim about cause. See Observational Studies and Experiments.

Capture and recapture estimates a population that cannot be listed. Tag 60 fish and release them. Later catch 80 fish and find 12 tagged. If the tagged fish have mixed fully, the same fraction of the lake is tagged: $\frac{60}{N}=\frac{12}{80}$60/N = 12/80, so $N=60\times 80/12=400$N = 60 × 80 / 12 = 400. See Capture and Recapture.

Why is the mean of a sample so well behaved?

Averaging makes the sample mean well behaved, because it cancels much of the chance variation in single values. Every sample of the same size gives a different mean, so the sample mean has its own distribution, the sampling distribution. See The Sampling Distribution. Its center is the population mean $\mu$μ, and its spread is the standard error $\frac{\sigma }{\sqrt{n}}$σ/√n, which shrinks as the sample grows. For large n its shape is close to normal, whatever the population looks like: that is the central limit theorem. Suppose heights have $\sigma =12$σ = 12 cm and you measure n = 36 people. The standard error is 12 ÷ 6 = 2 cm, since $\sqrt{36}=6$√36 = 6, so most sample means land within about 4 cm of $\mu$μ. The square root slows the gain: halving the standard error takes 4 times as many readings. Check with 144 people: since $\sqrt{144}=12$√144 = 12, the error is 12 ÷ 12 = 1 cm, half of 2.

Its mean is the population mean $\mu$μ, and its variance is $\frac{{\sigma }^2}{n}$σ²/n: n independent variances add to $n{\sigma }^2$nσ², and dividing the total by n divides the variance by $n^2$n². The square root, $\frac{\sigma }{\sqrt{n}}$σ/√n, is the standard error. See Mean and Variance of X-bar and Standard Error.

With $\sigma =20$σ = 20 and n = 100, the standard error is $20÷\sqrt{100}=2$20 ÷ √100 = 2. Individual values scatter by about 20, and their average by about 2.

The central limit theorem needs n to be large enough: about thirty is usually enough, and a badly lopsided population needs more. See The Central Limit Theorem and Applying the Theorem.

The same approximation works for the binomial when np and n(1 − p) are both greater than 5: a normal curve with mean np and variance np(1 − p) sits on top of the bars. The bars have width, so a count of 40 covers 39.5 to 40.5. See Approximating a Binomial and Continuity Correction.

How do you estimate a population value from a sample?

A population value is estimated by a single number, a point estimate, or by a range, a confidence interval. The sample mean x̄ is the point estimate of the population mean $\mu$μ, and it is unbiased: over many samples its average is exactly $\mu$μ. See Point Estimates. A 95% confidence interval adds and subtracts about two standard errors. If 100 bags have a mean mass of 502 g and $\sigma$σ is 10 g, the standard error is $10/10=1\text{g}$10 / 10 = 1 g, since $\sqrt{100}=10$√100 = 10, so the interval runs from about 500 g to 504 g. It works because about 95 in 100 sample means land within two standard errors of $\mu$μ, so about 95 in 100 such intervals contain $\mu$μ. When $\sigma$σ is unknown, estimate it from the sample, dividing by n − 1 rather than n, because data sit closer to their own mean than to $\mu$μ. Check the width: 504 − 500 = 4, which is four standard errors.

See Unbiased Sample Variance.

A confidence interval reports a range instead. Take the estimate, then add and subtract 1.96 standard errors, because z = 1.96 leaves 2.5 percent in each tail of the normal distribution.

The 95 percent describes the method: repeat it many times and 95 in 100 of the intervals contain the true mean. The one in front of you either contains it or does not. See Confidence Intervals.

How does a hypothesis test work?

A hypothesis test asks whether data are too surprising to blame on chance. Assume the null hypothesis $H_0$H₀, usually that nothing has changed, find how likely a result at least this extreme would be if $H_0$H₀ were true, and reject $H_0$H₀ if that probability, the p-value, is below a significance level chosen in advance, often 5%. Suppose a coin tossed 10 times gives 9 heads. If the coin is fair, the chance of 9 or more heads is 11 in 1,024, about 1%, so a one-tailed test at 5% rejects $H_0$H₀. A result that would be rare under $H_0$H₀ counts as evidence against it. A two-tailed test, chosen in advance, also counts 1 or fewer heads, so the p-value becomes 22 in 1,024, since 11 + 11 = 22, about 2%, which still rejects. Failing to reject never proves $H_0$H₀ true. Check the count: 9 heads happen in 10 ways and 10 heads in 1, so 10 + 1 = 11.

The null hypothesis, $H_0$H₀, claims no effect. The alternative, $H_1$H₁, claims an effect and needs evidence before you believe it. See Null and Alternative Hypotheses.

Decide before seeing the data which tail counts. A one-tailed test puts the whole 5 percent in one tail. A two-tailed test splits it, 2.5 percent in each tail, so its critical value sits further out. See One-Tailed and Two-Tailed.

The z-test handles a mean when $\sigma$σ is known: $z=\left(\bar{x}-\mu \right)/\left(\sigma ÷\sqrt{n}\right)$z = (x̄ − μ) / (σ ÷ √n). Subtract the claimed mean, divide by the standard error, and compare z with the critical value. See The Z-Test.

A critical region fixes the boundary in advance: mark off the most extreme 5 percent, and any result landing there rejects the null. A p-value shades everything at least as extreme as the result you saw and reports that area as a probability. See Critical Regions and P-Values.

For a proportion from a small sample, use the binomial directly: if a die is fair, sixes in 60 rolls follow $B\left(60,\frac{1}{6}\right)$B(60, 1/6), and the probability of 17 or more sixes is the p-value. See Testing a Proportion with the Binomial.

A type one error rejects a null hypothesis that was true, and its probability is the significance level you chose. A type two error fails to reject a false one. Demand more evidence and the first rate falls while the second rises. See Type One and Type Two Errors.

Which test fits which question?

The chi-squared tests compare observed counts with expected counts. In a two-way table, the expected count for a cell is the row total times the column total divided by the grand total. Square each gap between observed and expected, divide by the expected count, and add up every cell. The degrees of freedom are (rows − 1) × (columns − 1). See Expected Frequencies and The Chi-Squared Test for Independence.

The goodness of fit test uses the same statistic against a claimed distribution: a fair die rolled 60 times expects ten of each face. See The Chi-Squared Goodness of Fit Test.

When $\sigma$σ must be estimated from the sample, use t instead of z. The one-sample t-test divides the gap between the sample mean and the claimed mean by $s÷\sqrt{n}$s ÷ √n, with n − 1 degrees of freedom. The two-sample t-test divides the gap between two sample means by the standard error of that gap. See The One-Sample T-Test and The Two-Sample T-Test.

Spread can be tested too. Scale a sample variance by $\left(n-1\right)/{\sigma }^2$(n − 1) / σ² and it follows a chi-squared curve with n − 1 degrees of freedom. To compare two variances, divide the larger by the smaller; that ratio is F. See Testing a Variance with Chi-Squared and Comparing Two Variances with the F-Test.

The mistakes worth naming

• Reading a p-value backwards. It is the probability of the data given the null hypothesis, never the probability of the null hypothesis.
• Choosing the tail after seeing the data. That doubles the true false-alarm rate.
• Treating "not significant" as "no effect". A small sample often gives too little evidence to reject anything.
• Dividing by n for a sample variance. Divide by n − 1.
• Dropping the continuity correction. A count of 40 covers 39.5 to 40.5.
• Believing a big sample fixes a biased one. Size reduces random error and does nothing to systematic error.

Where this leads next

The binomial and normal distributions are built in the sets, counting and distributions guide. The mean and variance being tested come from the averages and spread guide. The test for a correlation coefficient is in the correlation and regression guide.

Related guides: Probability and Combined Events.