Roll two dice: a 4 appears with probability 11/36. Now someone tells you the sum was 6. The chance of a 4 jumps to 2/5. This chapter is about exactly that: conditional probability (what changes when we learn something), the multiplication rule for things that happen one after another, independence (when learning something changes nothing), and Bayes' theorem, which runs the reasoning backwards: from what we see to its likely cause.
News shrinks the world. Once we know F happened, only the outcomes inside F are still possible: F becomes the new sample space.
Read P(E|F) as “the probability of E given F”: the share of F where E also happens. It needs P(F) ≠ 0, and it works even when outcomes are not equally likely (Example 7).
When the outcomes are equally likely: count the outcomes in both E and F, and divide by the outcomes in F.
Toss three fair coins. E: at least two heads (P(E) = ½). F: the first coin shows tail. Given F, only THH, THT, TTH, TTT remain, and only THH is in E.
Properties, given F
The usual rules of probability still hold inside the new world F. For disjoint A and B the last term is 0, so just add.
P(A|B) from P(A ∩ B) and P(B).
Both boys given at least one boy.
Even card given more than 3.
Class XII given a girl.
4 on the third throw given 6, 5.
A 4 given sum 6.
Die > 4 given at least one tail.
For two things to both happen, the first must happen and then the second. So multiply their chances, one step at a time.
This is the definition of conditional probability turned around. E ∩ F (also written EF) means both E and F happen.
Each new factor is conditioned on everything that came before it. Four or more events work the same way.
Two black balls from 10 black and 5 white (Example 8). Drawing without replacement changes the second branch; with replacement it does not.
On a tree diagram
Two black balls, without replacement.
King, king, ace.
Two events are independent when learning that one happened does not change the chance of the other.
This says the same as P(E|F) = P(E) (and P(F|E) = P(F)): E and F are independent. If it fails, they are dependent.
Draw a card. A spade has probability ¼, and a spade given an ace is also ¼. Learning it is an ace did not change anything.
Independent is not mutually exclusive. Disjoint events with non-zero probabilities are never independent: if one happens, the other certainly did not.
If E and F are independent, so are E and F′, E′ and F, and E′ and F′.
Three events A, B, C are mutually independent when every pair multiplies and also
Two experiments are independent when every event of one is independent of every event of the other.
Multiple of 3 and even on a die.
Odd first throw, odd second throw.
Three coins, three pairs of events.
E, F independent ⇒ E, F′ independent.
At least one of A, B.
Sometimes we see a result and want its cause. Bayes' theorem runs the reasoning backwards: from what we saw to how likely each cause is.
The theorem of total probability, for any event A, when E₁, …, Eₙ form a partition of S. Written out: P(E₁)P(A|E₁) + P(E₂)P(A|E₂) + … + P(Eₙ)P(A|Eₙ).
E₁, …, Eₙ form a partition of S when they are pairwise disjoint, together make up S, and each has non-zero probability. So exactly one of them happens.
It reverses the condition: having seen A, how likely is each cause Eᵢ? The bottom is just P(A), by total probability. Published in 1763, after Thomas Bayes' death.
| Symbol | Name |
|---|---|
| Eᵢ | the hypotheses: the possible causes |
| P(Eᵢ) | the a priori (prior) probability, before we see A |
| P(Eᵢ|A) | the a posteriori (posterior) probability, after we see A |
How to use it
In the area model used throughout this page, each hypothesis is a column as wide as its prior. The shaded bottom of each column is P(A|Eᵢ), the total shaded area is P(A), and the posterior is one column's share of the shading.
Construction job with a possible strike.
Red ball: from Bag II?
Gold coin: other also gold?
Positive HIV test.
Defective bolt: machine B?
Late doctor: by train?
He says six: is it?
Four boxes; black ball from box III?
First to throw a six.
Two acceptable items: correct set-up?
A random variable is a real-valued function on the sample space: it gives each outcome a number. For example, X = number of heads in two tosses:
Another is Y = heads minus tails. Several random variables can be defined on the same sample space.
Fifteen questions to check yourself. Tap an option to check it and see the reasoning.
Every MCQ from Exercises 13.1–13.3 and the miscellaneous exercise, plus six concept checks.
Conditional probability
Multiplication rule
Independent events
That is, P(E|F) = P(E) and P(F|E) = P(F).
Total probability and Bayes
Both sums run over a partition E₁, …, Eₙ.
The earliest known count of chances in dice games appears in a commentary on the Divine Comedy.
His treatise on gambling, Liber de Ludo Aleae, counted favourable cases for two dice; it was published in 1663.
Explained why, with three dice, a sum of 10 is more likely than a sum of 9: more cases favour it.
Letters prompted by the gambler Chevalier de Méré laid the foundations of the science of probability.
Ars Conjectandi, published after his death, introduced the binomial distribution.
Built probability on axioms, as a function on sets, in Foundations of Probability.
Probability is easiest to see. Each walkthrough draws the right picture for its problem: a grid of equally likely outcomes (dice, coins, cards) where "given F" greys out everything outside F; a tree for things that happen one after another, with the probability of a path found by multiplying along it; a Venn diagram for rules about events; and an area model for Bayes' theorem, where each column is a possible cause and the shaded part is the chance of what we observed. Press Next step (or Play); each step is explained in detail under the controls.
P(E|F) = P(E ∩ F)/P(F): count (or weigh) the part of F where E also happens.
P(E ∩ F) = P(E) P(F|E): multiply along the branches of a tree.
E and F are independent when P(E ∩ F) = P(E) P(F).
Add the pieces to get P(A); a cause's share of those pieces is its posterior probability.
Bayes with a table, a game that could last for ever, and Bayes with repeated trials.
Your email app deciding whether a message is spam, a class where two people share a birthday, a website with backup servers, the odds of being late when you have two ways to school, and a board game won by the first six: each is conditional probability, independence or Bayes' theorem at work.
The word "prize" appears in 40% of spam emails but only 2% of normal ones. If a fraction s of all your email is spam, what is the chance that an email containing "prize" is spam? Bayes: P(spam | "prize") = 0.4s/(0.4s + 0.02(1 − s)).
The chance that n people all have different birthdays is (365/365) × (364/365) × (363/365) × … × ((366 − n)/365): each new person must avoid the birthdays already taken. It falls surprisingly fast: with just 23 people a shared birthday is more likely than not.
Each server is up with probability p, independently of the others. The site works if at least one server is up: P(works) = 1 − (1 − p)ⁿ (Example 14 with n events). Watch how fast extra servers push reliability towards 1.
You take the bus on a fraction b of days and cycle otherwise. The bus makes you late 30% of the time, cycling 10%. Total probability gives how often you are late; Bayes tells your teacher, from the fact that you were late, how likely it is that you came by bus.
Two players take turns; the first to succeed wins. If each turn succeeds with probability s, the first player wins with probability s/(1 − (1 − s)²), from the geometric series. With a die (s = 1/6) that is 6/11. The bars show how the win splits over the first few rounds.
Decide first, then tap each card.
Every question that asks you to find, check or show something has an animated walkthrough here; the 9 multiple-choice questions are in the Lesson's self-check. Pick a question from the chips above each player, then press Play or step through with Next. Questions with several parts that need different pictures are split into separate chips.
Formulas, coins, dice, cards, families and a question bank. (Q16–Q17 are in the quiz.)
Draws without replacement, product tests for independence, and "at least one" by the complement. (Q17–Q18 are in the quiz.)
Bags, students, tests, coins, drivers, machines and a lost card. (Q13–Q14 are in the quiz.)
Subsets, families, grey hair, right-handers, a leap year, marbles, heart attacks, determinants, subsystems and a transferred ball. (Q11–Q13 are in the quiz.)
This chapter builds on eight ideas from Classes 10 and 11: events as sets, counting outcomes, probability as "favourable over total", the rules of probability, fraction arithmetic, tree diagrams, Venn diagrams and infinite geometric series. Here is each one from its roots. Every example and exercise links here exactly where you need it, and a button takes you straight back to the problem.
The sample space S is the set of all possible outcomes (for a die, {1, 2, 3, 4, 5, 6}). An event is any subset: "even" is {2, 4, 6}.
| Symbol | In words |
|---|---|
| A ∩ B | “A and B”: both happen |
| A ∪ B | “A or B”: at least one happens |
| A′ | “not A” |
| A ∩ B = φ | no common outcome: A and B are mutually exclusive |
If a first stage can go m ways and a second n ways, together they go m × n ways: the multiplication principle.
The last tile counts the ways of choosing r things from n when order does not matter.
If all n(S) outcomes are equally likely, each has probability 1/n(S).
An event with n(E) outcomes has this probability. Why believe it? Repeat the experiment many times: the fraction of times E happens settles down near P(E).
For example a coin, then a die. Give each outcome its own probability and add the probabilities of the outcomes in E instead of counting them.
Every probability is between 0 and 1, with P(S) = 1 and P(φ) = 0.
True for any two events. Adding P(A) and P(B) counts the overlap twice, so subtract it once. For mutually exclusive events just add.
Often the fastest route: “at least one” = 1 − “none”.
Conditional probabilities are fractions divided by fractions: (4/13)/(9/13). Rule: (a/b) ÷ (c/d) = (a/b) × (d/c). When both have the same denominator, it simply cancels: (4/13)/(9/13) = 4/9. Multiplying probabilities (multiplication rule) multiplies tops and bottoms; adding them needs a common denominator.
When an experiment happens in stages, draw one branching point per stage. Label every branch with its probability given everything before it; the branches from one point add up to 1. The probability of a whole path is the product of its branches, and the probability of an event made of several paths is the sum of those products. All the path products together add up to 1.
Draw S as a box and A and B as overlapping circles. They cut S into four regions.
| Region | Event |
|---|---|
| A only | A ∩ B′ |
| both | A ∩ B |
| B only | A′ ∩ B |
| neither | A′ ∩ B′ |
Both read straight off the picture.
A game that could last for ever (Example 23) needs an infinite sum.
This only works when |r| < 1: then the terms shrink fast enough for the partial sums to settle on a limit.
Why it works
Leave your email and we will write when a new chapter or a new set of worked solutions is ready. That is all we use it for, and you can ask to be taken off the list at any time.
You are on the list. We will email you when new chapters are added, and for nothing else.