Sets, Counting and Distributions

☰ Contents

Probability starts by counting outcomes, and it stops working once there are too many to list. Two things replace listing: counting rules that give a total without writing it out, and distributions that cover a whole family of situations.

Why bother with set notation?

Set notation says exactly which outcomes you mean, where English is ambiguous: "tea or coffee" can exclude having both, but the union of two sets includes it. A set is a collection of distinct objects, and the universal set is everything under discussion. Take the whole numbers 1 to 10 as the universal set, with A = {2, 4, 6, 8, 10} and B = {1, 2, 3, 4}. Then A ∪ B, in A or B or both, is {1, 2, 3, 4, 6, 8, 10}, and A ∩ B, in both, is {2, 4}. The complement A', everything not in A, is {1, 3, 5, 7, 9}. Each symbol names a region of a Venn diagram, which turns probability questions into counts. Adding the sizes of A and B counts the overlap twice, so subtract it once. Check with the lists: 5 + 4 − 2 = 7, the size of the union.

973

the intersection is counted in both totals, so it is subtracted once: |A ∪ B| = |A| + |B| − |A ∩ B|

Make |A ∪ B| = 16

UAB2233
Everything in either circle is the union, A ∪ B: in A, in B, or in both. Full lesson: Set Notation

Besides the intersection A ∩ B and the complement, B is a subset of A when every element of B is also in A. See Set Notation, Venn Diagrams and Complements and Subsets.

In symbols, n(A ∪ B) = n(A) + n(B) − n(A ∩ B).

UAB7540
So subtract the overlap once: n(A ∪ B) = 12 + 9 − 5 = 16. Full lesson: Counting a Union

A probability is then a count of regions divided by the count of the whole, and a condition narrows which regions count. See Counting a Union, Probability in Set Notation and Conditional Probability from Tables.

Some infinities are strictly larger: the decimals between 0 and 1 cannot be listed even in an endless list. See Cantor's Diagonal: Bigger Infinities.

diagonal 4, 7, 3the new decimal is 0.584change each: 5, 8, 4it differs from row n at place n0.584 is on no row
Change every diagonal digit: 0.584 differs from every row, so this list has missed it. Full lesson: Cantor’s Diagonal: Bigger Infinities

Now you

UAB439

n(A) = 7, n(B) = 12, and 3 are in both. What is n(A ∪ B)?

UAB677

n(A) = 13, n(B) = 14, and 7 are in both. What is n(A ∪ B)?

How do you count without listing?

To count without listing, multiply the numbers of choices made one after another, and add the numbers of choices that are alternatives. A restaurant offers 3 starters, 4 mains and 2 desserts. A meal of one of each can be chosen in 3 × 4 × 2 = 24 ways, since each starter pairs with every main, and each pair with every dessert. A single dish from the starters or the desserts can be chosen in 3 + 2 = 5 ways. Multiplying works because every branch of the choice tree splits the same way. The count shrinks when each choice uses up an option: 4 different books can be shelved in 4 × 3 × 2 × 1 = 24 orders, written 4!. When order does not matter, divide by the number of orders in which each selection was counted. Listing confirms small counts: 2 drinks and 2 cakes give 2 × 2 = 4 pairs.

3 × 5 = 15

multiplication is commutative: 3 × 5 = 5 × 3

Make 24 as a rectangle, then find another factor pair

teacakesconetartbuncoffeecakesconetartbunjuicecakesconetartbun
There are three drinks, then four cakes for each. Each drink opens four more paths. Full lesson: Adding and Multiplying Choices

Three starters and four mains give 12 two-course meals; three starters or four mains, one course only, gives 7. See Adding and Multiplying Choices.

Arranging n different things in a row can be done in n! ways: n choices for the first position, n − 1 for the second, down to 1. There is one way to arrange nothing, so 0! = 1. See Factorials.

5×4×3×2×15! = 120
The exclamation mark means multiply all the way down to 1, so 5 slots give 120 ways. Full lesson: Factorials

Permutations or combinations?

Ask whether reordering the same selection produces a different outcome. If it does, count permutations; if it does not, count combinations.

A permutation fills r positions from n candidates in order: nPr = n!/(n − r)!. A combination has no order: nCr = n!/(r!(n − r)!). Eight runners give 336 orderings of three medals but only 56 teams of three. See Permutations and Combinations.

8×7×6336 ordered picks3! = 6 → 56
Every team was counted 3! = 6 times, once for each order of picking. Full lesson: Combinations

When items repeat, divide by the factorial of each repeat: BANANA arranges in 6! ÷ (3!2!) = 60 ways. When two people must sit together, glue them into one item and multiply by the 2 ways they can sit inside. See Repeats and Restrictions.

A triangle is an unordered choice of three vertices, so 10 points with no three in a line give 120 triangles. See Counting with Geometry.

Now you

6C3what does this come to?

How many ways to choose a team of 3 from 6?

6C2what does this come to?

How many ways to choose a team of 2 from 6?

What is a random variable?

A random variable attaches a number to every outcome of an experiment. Toss two fair coins and let X be the number of heads: X takes the values 0, 1 and 2 with probabilities 1/4, 1/2 and 1/4, because HH, HT, TH and TT are equally likely and two of them give one head. See Random Variables. The probabilities of a discrete variable add to 1, and here 1/4 + 1/2 + 1/4 = 1. Its expected value, E(X), is the long-run mean: multiply each value by its probability and add, so 0 × 1/4 + 1 × 1/2 + 2 × 1/4 = 1 head. Naming the variable turns a question about outcomes into arithmetic. A continuous variable, such as a height, needs a different method: probabilities are areas under a curve, and one exact value has probability 0. By symmetry, half of 2 coins land heads on average: 2 ÷ 2 = 1.

A probability distribution lists every value X can take with the probability of each. See Probability Distributions.

2/10 + 3/10 + 5/10 = 1
The probabilities must total 1, so a missing one is 1 minus the rest. Full lesson: Probability Distributions

E(X) is a long-run average per trial, not a prediction of one trial. See Expected Value.

0102030405001050E(X) = 9
E(X) = 9 lands between the prizes, just short of 10. Nobody ever wins 9. Full lesson: Expected Value

Variance measures spread as the expected squared distance from the mean, usually computed as E(X²) − [E(X)]². Squaring stops distances above and below the mean canceling. See Variance.

Changing units changes the two differently: E(aX + b) = aE(X) + b, but Var(aX + b) = a²Var(X), because variance is measured in squared units. See Transforming a Random Variable.

xy
A shift leaves the spread alone; doubling quadruples it: Var(2X) = 4Var(X). Full lesson: Transforming a Random Variable

Now you

014

X is 0 with probability 1/2 and 14 with probability 1/2. What is E(X)?

06

X is 0 with probability 1/2 and 6 with probability 1/2. What is E(X)?

What is the binomial distribution for?

The binomial distribution counts the successes in a fixed number n of independent trials, each with the same probability p of success. One such trial is a Bernoulli trial. See Bernoulli Trials. The count X is written X ~ B(n, p), and the chance of exactly r successes is the number of orders, nCr, times the chance of one order. Roll a die 4 times and count sixes. The chance of exactly one six is 4 × 1/6 × 125/216 = 125/324: the six can come on any of the 4 rolls, and the other three miss with probability 5/6 each. The factor nCr appears because every order of the successes has the same probability. Four conditions must hold: a fixed number of trials, two outcomes, a constant p, and independence. Cards drawn without replacement break the constant p, so they need another model. Check with the mean: E(X) = np, and 4 × 1/6 = 2/3 of a six per four rolls.

05101520n = 8, p = 0.15np = 1.2

np < 5: the distribution is still skewed toward 0, so the Normal curve is not yet a safe stand-in for it; np = 1.2

Increase n until np ≥ 5

The notation is X ~ B(n, p), and P(X = r) = nCr × p^r × (1 − p)^(n − r). See The Binomial Distribution.

P(X = r)r successes
and the rest failures
times the ways to arrange them
Choose which r trials succeed and multiply p for each success, 1 − p for each failure. Full lesson: The Binomial Distribution

The mean and variance need no summing: E(X) = np and Var(X) = np(1 − p), so twenty tosses of a fair coin have mean 10 and variance 5. See Binomial Mean and Variance.

012345
Take 5 trials with probability 1/2. The mean is np = 2.5, right under the peak. Full lesson: Binomial Mean and Variance

Independence is the condition that fails most often: twenty people from the same household are not twenty independent trials. See Modeling with a Binomial.

if faults clusterone fault makes the next more likelynot independent
If a bad batch makes faults come in clusters, the trials are not independent and the model no longer holds. Full lesson: Modeling with a Binomial

Now you

4C22 successes
and 2 failures

X ~ B(4, 1/2). What is P(X = 2)?

5C2what does this come to?

In how many ways can exactly 2 of 5 trials succeed?

How does the normal distribution work?

The normal distribution describes a continuous quantity that clusters symmetrically around its mean μ, with spread set by the standard deviation σ. Probability is area under its bell-shaped curve, and the total area is 1. See Continuous Variables. About 68% of values lie within one standard deviation of the mean, and about 95% within two. For other areas, standardize: subtract the mean and divide by σ, which turns any normal distribution into the standard normal, with mean 0 and standard deviation 1. If exam marks have μ = 60 and σ = 8, a mark of 76 is (76 − 60) ÷ 8 = 2 standard deviations above the mean, so about 1 mark in 44 lies above it. Standardizing works because every normal curve has the same shape, only stretched and shifted. One exact value has probability 0, so questions ask about ranges, and a negative z uses symmetry. Undo the standardizing as a check: 60 + 2 × 8 = 76.

xy
A probability is the area over a range of values, so one exact value has none. Full lesson: Continuous Variables

Three standard deviations, σ each, either side of the mean hold 99.7% of the values. See The Normal Distribution.

xy
About 68 percent lies within one standard deviation, sigma, of the mean, either side. Full lesson: The Normal Distribution

There is a different curve for every mean and standard deviation, so tables are printed only for the standard normal, which z = (x − μ)/σ reaches from any of them. See Standardizing.

A z-score is a change of units, not a change of data, which is what makes comparison possible: a z of 1.8 on a chemistry paper beats a z of 1.2 on a history paper.

A z-table gives the area to the left of a z-score: subtract it from 1 for a right-hand tail, subtract the smaller from the larger for a band, and use symmetry for a negative z. An inverse normal question runs the table backward: the top 10% starts where 90% of the area lies to the left, so z = 1.28, then undo the standardizing with x = μ + zσ. See Reading the Z-Table, Reading Normal Probabilities and Inverse Normal.

Sums and differences of independent normal variables are normal again, and their means add. See Combining Normals.

xy
Var(X − Y) = 1 + 1 = 2 as well — independent spreads add even when you subtract. Full lesson: Combining Normals

Now you

area to the leftz = 1.280.8997z = 1.6450.9500z = 1.960.9750z = 2.330.9901z = 2.580.9951

What is P(2.33 < Z < 2.58)?

area to the leftz = 1.280.8997z = 1.6450.9500z = 1.960.9750z = 2.330.9901z = 2.580.9951

What is P(1.645 < Z < 1.96)?

The mistakes worth naming

Where this leads next

Everything here describes a distribution you assume. Deciding from data which assumption is defensible is the sampling and inference guide. The counting rules feed the tree diagrams in the probability guide, and the combination numbers are the coefficients in the binomial theorem.

The lessons are listed in Sets and Counting and Distributions. Related guides: Logic and Proof.

Your turn

Three to try — tap what you get.

|A| = 5, |B| = 4, |A ∩ B| = 2. What is |A ∪ B|?

How many ways can 4 books line up on a shelf?

3 shirts and 2 pairs of pants make how many outfits?

Questions and answers

When do you use permutations instead of combinations?
Use permutations when the order of the selection matters and combinations when it does not. Choosing a president, secretary and treasurer from a club is a permutation, because swapping two of the people gives a different outcome. Choosing three members for a committee is a combination, because the same three people are the same committee however they were listed. The two are related: the combination count is the permutation count divided by the number of orderings, which is r factorial.
What does expected value mean if you can never get it?
Expected value is the long-run average outcome per trial, not a value the variable will ever take. A fair die has an expectation of 3.5, which no roll produces, but a thousand rolls will total close to 3,500. It is a balance point for the distribution, and it is the right quantity whenever outcomes will be repeated and totaled, which is why insurers and casinos work in expectations rather than in typical results.
When can you model something with a binomial distribution?
When there is a fixed number of trials, each trial has exactly two outcomes, the probability of success is the same every time, and the trials are independent. Twenty free throws by the same player fit reasonably well. Drawing twenty cards without replacement does not, because the probability changes each draw, and surveying twenty friends does not, because their answers are unlikely to be independent.
What is standardizing and why does it help?
Standardizing converts a value into the number of standard deviations it sits from the mean, by subtracting the mean and dividing by the standard deviation. The result is a z-score, and it removes the units, so every normal distribution collapses onto one standard shape. That is what makes a single table usable for every problem, and it also lets you compare a score on one test with a score on a completely different one.
What is the difference between a discrete and a continuous distribution?
A discrete distribution assigns a probability to each individual value, so asking for the probability of exactly 3 is meaningful. A continuous distribution assigns probability to intervals through the area under a curve, and the probability of any exact value is zero. That is why continuous questions are always phrased as less than, greater than or between, and why the areas under the whole curve must total one.
What does the inverse normal do?
It runs the normal table backwards: instead of giving a value and asking for a probability, you give a probability and ask for the value. It answers questions like what mark puts a student in the top ten percent, or what weight limit fails only two percent of items. Find the z-score for the required area first, then convert it back to the original units by multiplying by the standard deviation and adding the mean.
Mr. Chalk