Sets, Counting and Distributions
☰ Contents
Probability starts by counting outcomes, and it stops working once there are too many to list. Two things replace listing: counting rules that give a total without writing it out, and distributions that cover a whole family of situations.
Why bother with set notation?
Set notation says exactly which outcomes you mean, where English is ambiguous: "tea or coffee" can exclude having both, but the union of two sets includes it. A set is a collection of distinct objects, and the universal set is everything under discussion. Take the whole numbers 1 to 10 as the universal set, with A = {2, 4, 6, 8, 10} and B = {1, 2, 3, 4}. Then , in A or B or both, is {1, 2, 3, 4, 6, 8, 10}, and , in both, is {2, 4}. The complement A', everything not in A, is {1, 3, 5, 7, 9}. Each symbol names a region of a Venn diagram, which turns probability questions into counts. Adding the sizes of A and B counts the overlap twice, so subtract it once. Check with the lists: 5 + 4 − 2 = 7, the size of the union.
the intersection is counted in both totals, so it is subtracted once: |A ∪ B| = |A| + |B| − |A ∩ B|
Make |A ∪ B| = 16
Besides the intersection and the complement, B is a subset of A when every element of B is also in A. See Set Notation, Venn Diagrams and Complements and Subsets.
In symbols, .
A probability is then a count of regions divided by the count of the whole, and a condition narrows which regions count. See Counting a Union, Probability in Set Notation and Conditional Probability from Tables.
Some infinities are strictly larger: the decimals between 0 and 1 cannot be listed even in an endless list. See Cantor's Diagonal: Bigger Infinities.
Now you
n(A) = 7, n(B) = 12, and 3 are in both. What is ?
n(A) = 13, n(B) = 14, and 7 are in both. What is ?
How do you count without listing?
To count without listing, multiply the numbers of choices made one after another, and add the numbers of choices that are alternatives. A restaurant offers 3 starters, 4 mains and 2 desserts. A meal of one of each can be chosen in 3 × 4 × 2 = 24 ways, since each starter pairs with every main, and each pair with every dessert. A single dish from the starters or the desserts can be chosen in 3 + 2 = 5 ways. Multiplying works because every branch of the choice tree splits the same way. The count shrinks when each choice uses up an option: 4 different books can be shelved in 4 × 3 × 2 × 1 = 24 orders, written 4!. When order does not matter, divide by the number of orders in which each selection was counted. Listing confirms small counts: 2 drinks and 2 cakes give 2 × 2 = 4 pairs.
multiplication is commutative: 3 × 5 = 5 × 3
Make 24 as a rectangle, then find another factor pair
Three starters and four mains give 12 two-course meals; three starters or four mains, one course only, gives 7. See Adding and Multiplying Choices.
Arranging n different things in a row can be done in n! ways: n choices for the first position, n − 1 for the second, down to 1. There is one way to arrange nothing, so 0! = 1. See Factorials.
Permutations or combinations?
Ask whether reordering the same selection produces a different outcome. If it does, count permutations; if it does not, count combinations.
A permutation fills r positions from n candidates in order: . A combination has no order: . Eight runners give 336 orderings of three medals but only 56 teams of three. See Permutations and Combinations.
When items repeat, divide by the factorial of each repeat: BANANA arranges in 6! ÷ (3!2!) = 60 ways. When two people must sit together, glue them into one item and multiply by the 2 ways they can sit inside. See Repeats and Restrictions.
A triangle is an unordered choice of three vertices, so 10 points with no three in a line give 120 triangles. See Counting with Geometry.
Now you
How many ways to choose a team of 3 from 6?
How many ways to choose a team of 2 from 6?
What is a random variable?
A random variable attaches a number to every outcome of an experiment. Toss two fair coins and let X be the number of heads: X takes the values 0, 1 and 2 with probabilities , and , because HH, HT, TH and TT are equally likely and two of them give one head. See Random Variables. The probabilities of a discrete variable add to 1, and here . Its expected value, E(X), is the long-run mean: multiply each value by its probability and add, so head. Naming the variable turns a question about outcomes into arithmetic. A continuous variable, such as a height, needs a different method: probabilities are areas under a curve, and one exact value has probability 0. By symmetry, half of 2 coins land heads on average: 2 ÷ 2 = 1.
A probability distribution lists every value X can take with the probability of each. See Probability Distributions.
E(X) is a long-run average per trial, not a prediction of one trial. See Expected Value.
Variance measures spread as the expected squared distance from the mean, usually computed as . Squaring stops distances above and below the mean canceling. See Variance.
Changing units changes the two differently: E(aX + b) = aE(X) + b, but VarVar(X), because variance is measured in squared units. See Transforming a Random Variable.
Now you
X is 0 with probability and 14 with probability . What is E(X)?
X is 0 with probability and 6 with probability . What is E(X)?
What is the binomial distribution for?
The binomial distribution counts the successes in a fixed number n of independent trials, each with the same probability p of success. One such trial is a Bernoulli trial. See Bernoulli Trials. The count X is written X ~ B(n, p), and the chance of exactly r successes is the number of orders, nCr, times the chance of one order. Roll a die 4 times and count sixes. The chance of exactly one six is : the six can come on any of the 4 rolls, and the other three miss with probability each. The factor nCr appears because every order of the successes has the same probability. Four conditions must hold: a fixed number of trials, two outcomes, a constant p, and independence. Cards drawn without replacement break the constant p, so they need another model. Check with the mean: E(X) = np, and of a six per four rolls.
np < 5: the distribution is still skewed toward 0, so the Normal curve is not yet a safe stand-in for it; np = 1.2
Increase n until np ≥ 5
The notation is X ~ B(n, p), and . See The Binomial Distribution.
The mean and variance need no summing: E(X) = np and Var(X) = np(1 − p), so twenty tosses of a fair coin have mean 10 and variance 5. See Binomial Mean and Variance.
Independence is the condition that fails most often: twenty people from the same household are not twenty independent trials. See Modeling with a Binomial.
Now you
X ~ . What is P(X = 2)?
In how many ways can exactly 2 of 5 trials succeed?
How does the normal distribution work?
The normal distribution describes a continuous quantity that clusters symmetrically around its mean , with spread set by the standard deviation . Probability is area under its bell-shaped curve, and the total area is 1. See Continuous Variables. About 68% of values lie within one standard deviation of the mean, and about 95% within two. For other areas, standardize: subtract the mean and divide by , which turns any normal distribution into the standard normal, with mean 0 and standard deviation 1. If exam marks have and , a mark of 76 is (76 − 60) ÷ 8 = 2 standard deviations above the mean, so about 1 mark in 44 lies above it. Standardizing works because every normal curve has the same shape, only stretched and shifted. One exact value has probability 0, so questions ask about ranges, and a negative z uses symmetry. Undo the standardizing as a check: 60 + 2 × 8 = 76.
Three standard deviations, each, either side of the mean hold 99.7% of the values. See The Normal Distribution.
There is a different curve for every mean and standard deviation, so tables are printed only for the standard normal, which reaches from any of them. See Standardizing.
A z-table gives the area to the left of a z-score: subtract it from 1 for a right-hand tail, subtract the smaller from the larger for a band, and use symmetry for a negative z. An inverse normal question runs the table backward: the top 10% starts where 90% of the area lies to the left, so z = 1.28, then undo the standardizing with . See Reading the Z-Table, Reading Normal Probabilities and Inverse Normal.
Sums and differences of independent normal variables are normal again, and their means add. See Combining Normals.
Now you
What is P(2.33 < Z < 2.58)?
What is P(1.645 < Z < 1.96)?
The mistakes worth naming
- Using a permutation where order does not matter. The same three people are the same committee however they were listed.
- Using a binomial when p changes. Drawing cards without replacement changes p at every draw.
- Subtracting variances for a difference. For independent variables, Var(X − Y) = Var(X) + Var(Y).
- Asking for P(X = 5) from a normal distribution. An exact value has probability 0, so ask about an interval.
Where this leads next
Everything here describes a distribution you assume. Deciding from data which assumption is defensible is the sampling and inference guide. The counting rules feed the tree diagrams in the probability guide, and the combination numbers are the coefficients in the binomial theorem.
The lessons are listed in Sets and Counting and Distributions. Related guides: Logic and Proof.
Your turn
Three to try — tap what you get.
|A| = 5, |B| = 4, |A ∩ B| = 2. What is |A ∪ B|?
How many ways can 4 books line up on a shelf?
3 shirts and 2 pairs of pants make how many outfits?
Questions and answers
- When do you use permutations instead of combinations?
- Use permutations when the order of the selection matters and combinations when it does not. Choosing a president, secretary and treasurer from a club is a permutation, because swapping two of the people gives a different outcome. Choosing three members for a committee is a combination, because the same three people are the same committee however they were listed. The two are related: the combination count is the permutation count divided by the number of orderings, which is r factorial.
- What does expected value mean if you can never get it?
- Expected value is the long-run average outcome per trial, not a value the variable will ever take. A fair die has an expectation of 3.5, which no roll produces, but a thousand rolls will total close to 3,500. It is a balance point for the distribution, and it is the right quantity whenever outcomes will be repeated and totaled, which is why insurers and casinos work in expectations rather than in typical results.
- When can you model something with a binomial distribution?
- When there is a fixed number of trials, each trial has exactly two outcomes, the probability of success is the same every time, and the trials are independent. Twenty free throws by the same player fit reasonably well. Drawing twenty cards without replacement does not, because the probability changes each draw, and surveying twenty friends does not, because their answers are unlikely to be independent.
- What is standardizing and why does it help?
- Standardizing converts a value into the number of standard deviations it sits from the mean, by subtracting the mean and dividing by the standard deviation. The result is a z-score, and it removes the units, so every normal distribution collapses onto one standard shape. That is what makes a single table usable for every problem, and it also lets you compare a score on one test with a score on a completely different one.
- What is the difference between a discrete and a continuous distribution?
- A discrete distribution assigns a probability to each individual value, so asking for the probability of exactly 3 is meaningful. A continuous distribution assigns probability to intervals through the area under a curve, and the probability of any exact value is zero. That is why continuous questions are always phrased as less than, greater than or between, and why the areas under the whole curve must total one.
- What does the inverse normal do?
- It runs the normal table backwards: instead of giving a value and asking for a probability, you give a probability and ask for the value. It answers questions like what mark puts a student in the top ten percent, or what weight limit fails only two percent of items. Find the z-score for the required area first, then convert it back to the original units by multiplying by the standard deviation and adding the mean.