Probability Theory
Sample space, Ω is a set of all possible outcomes of the random experiment (therefore, its probability
equals 1). Events 𝐴, 𝐵, … are subsets, {… , … } , of the sample space. Probability is denoted as 𝑃 and the
probability that event 𝐴 will occur as 𝑃(𝐴). The probability of any event must be positive and is at most
1. Hence: 0 ≤ 𝑃(𝐴) ≤ 1 for all events 𝐴. These sets are represented with a Venn diagram. This diagram
can be useful to understand what is asked. There are single events, multiple events, and empty events,
∅ (an impossible event, with a probability of 0%).
Dependent (one influences the probability of other events)
A and B both occur dependent 𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴) ∗ 𝑃(𝐵|𝐴) = 𝑃(𝐵) ∗ 𝑃(𝐴|𝐵)
A and B and C both occur dependent 𝑃(𝐴 ∩ 𝐵 ∩ 𝐶) = 𝑃(𝐴) ∗ 𝑃(𝐵|𝐴) ∗ 𝑃(𝐶|𝐴 ∩ 𝐵)
A occur given that B 𝑃(𝐴 ∩ 𝐵) 𝑃(𝐴)𝑃(𝐵|𝐴)
𝑃(𝐴|𝐵) = =
𝑃(𝐵) 𝑃(𝐵)
Independent (events do not affect one another)
A and B both occur independent 𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴) ∗ 𝑃(𝐵)
A does occur but B does not 𝑃(𝐴 ∩ 𝐵𝐶 ) = 𝑃(𝐴) ∗ (1 − 𝑃(𝐵))
A occur given that B 𝑃(𝐴|𝐵) = 𝑃(𝐴)
A and/or B occur jointly (outcomes overlap) 𝑃(𝐴 ∪ 𝐵) = 𝑃(𝐴) + 𝑃(𝐵) − 𝑃(𝐴 ∩ 𝐵)
Disjoint events (no outcomes overlap since events do not occur at the same time a.k.a. mutually exclusive events)
A and/or B occur disjoint 𝑃(𝐴 ∪ 𝐵) = 𝑃(𝐴) + 𝑃(𝐵)
A and B occur disjoint 𝑃(𝐴 ∩ 𝐵) = ∅ = 0
A and B do not occur jointly 𝑃(𝐴 ∩ 𝐵)𝐶 = 1 − 𝑃(𝐴 ∩ 𝐵)
Do not occur
A does not occur 𝑃(𝐴𝐶 ) = 1 − 𝑃(𝐴)
None of the three events A, B and C occurs 𝑃(𝐴𝐶 ∩ 𝐵𝐶 ∩ 𝐶 𝐶 )
Whenever TI-30XB MultiView Texas Instruments displays fractions, exact square root or exact pi answers, then press [⊲⊳] to
toggle the display to decimal answers.
Random Variables
A random variable (rv), denoted as 𝑋, is a prescript for measuring some feature of interest of the
sample space; it assigns a value (usually numbers) to each outcome of the experiment, but its actual
outcome is not known in advance (random) and is determined by chance. The actual outcome of a
random variable is called the realisation, denoted as 𝑥, small letters.
Note: A random variable is assumed independent if you are not given more information.
There are two types of random variables: quantitative (ordinary numbers as values) and qualitative
(categories as values). A random variable can be discrete or continuous. Discrete variables have finite
or countable number of outcomes. Continuous variables can take on any value in an interval.
Probability Density Function
Probability Density Function (pdf), 𝑓 shows the probabilities for each possible outcome of a discrete
random variable. Hence: 𝑓(𝑥) = 𝑃(𝑋 = 𝑥) for all outcomes 𝑥 of 𝑋. Two properties:
i. 𝑓(𝑥) ≥ 0 𝑓𝑜𝑟 𝑎𝑙𝑙 𝑥
ii. total surface equals 1
, (Cumulative) Distribution Function
(Cumulative) Distribution Function (cdf), F, Φ always adds up the probabilities per outcome.
Two definitions:
• cdf discrete rv: 𝐹(𝑎) = 𝑃(𝑋 ≤ 𝑎) for all real numbers 𝑎
• cdf continuous rv: 𝐹(𝑎) = 𝑃(𝑋 ≤ 𝑎) = 𝑃(𝑋 < 𝑎) for all real numbers 𝑎
Important properties:
• 𝐹 is non-decreasing
• 𝐹(−∞) = 0; 𝐹(∞) = 1
• 𝐹(𝑏) − 𝐹(𝑎) = 𝑃(𝑋 ≤ 𝑏) − 𝑃(𝑋 ≤ 𝑎) = 𝑃(𝑎 < 𝑋 ≤ 𝑏) for all 𝑎 and 𝑏
Note: Rounded brackets do not include given numbers, in which squared brackets do. Hence: 𝑃([35, ∞)) = 𝑃(𝑋 ≥ 35).
The table of the pdf and cdf:
𝑥 0 1 2
𝑓(𝑥) 0.25 0.50 0.25
𝐹(𝑥) 0.25 0.75 1
Four scenarios:
1. 𝑃(𝑋 > 1) = 𝑃(𝑋 = 2) = 0.25
2. 𝑃(𝑋 < 2) = 𝑃(𝑋 = 0) + 𝑃(𝑋 = 1) = 1 − 𝑃(𝑋 = 2) = 0.75
3. 𝑃(𝑋 ≥ 0) = 𝑃(𝑋 = 0) + 𝑃(𝑋 = 1) + 𝑃(𝑋 = 2) = 1
4. 𝑃(𝑋 ≤ 1) = 𝑃(𝑋 = 0) + 𝑃(𝑋 = 1) = 0.75
The graph of the cdf of 𝑋: left continuous and right discrete.
Expectation, Variance and Standard Deviation
Chance is driven by the probability distribution of 𝑋. The actual outcome will deviate from that
expectation.
Expectation, 𝐸(𝑋): 𝜇𝑥 = ∑𝑥 𝑥 ∗ 𝑓(𝑥) Note: Also called expected value, (weighted) average or mean.
Variance, 𝑉(𝑋): 𝜎 2 = ∑𝑥(𝑥 − 𝜇𝑥 )2 ∗ 𝑓(𝑥) Note: The variance is always non-negative. Hence: 𝑉(𝑋) ≥ 0
2
Shortcut formula: 𝜎 2 = (∑𝑥 𝑥 2 ∗ 𝑓(𝑥)) − 𝜇𝑥2 = 𝐸(𝑋 2 ) − (𝐸(𝑋))
Standard Deviation, 𝑆𝐷(𝑋): 𝜎 = √𝑉(𝑋) Note: Also called mean deviation from averages or mean spread.
Sample space, Ω is a set of all possible outcomes of the random experiment (therefore, its probability
equals 1). Events 𝐴, 𝐵, … are subsets, {… , … } , of the sample space. Probability is denoted as 𝑃 and the
probability that event 𝐴 will occur as 𝑃(𝐴). The probability of any event must be positive and is at most
1. Hence: 0 ≤ 𝑃(𝐴) ≤ 1 for all events 𝐴. These sets are represented with a Venn diagram. This diagram
can be useful to understand what is asked. There are single events, multiple events, and empty events,
∅ (an impossible event, with a probability of 0%).
Dependent (one influences the probability of other events)
A and B both occur dependent 𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴) ∗ 𝑃(𝐵|𝐴) = 𝑃(𝐵) ∗ 𝑃(𝐴|𝐵)
A and B and C both occur dependent 𝑃(𝐴 ∩ 𝐵 ∩ 𝐶) = 𝑃(𝐴) ∗ 𝑃(𝐵|𝐴) ∗ 𝑃(𝐶|𝐴 ∩ 𝐵)
A occur given that B 𝑃(𝐴 ∩ 𝐵) 𝑃(𝐴)𝑃(𝐵|𝐴)
𝑃(𝐴|𝐵) = =
𝑃(𝐵) 𝑃(𝐵)
Independent (events do not affect one another)
A and B both occur independent 𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴) ∗ 𝑃(𝐵)
A does occur but B does not 𝑃(𝐴 ∩ 𝐵𝐶 ) = 𝑃(𝐴) ∗ (1 − 𝑃(𝐵))
A occur given that B 𝑃(𝐴|𝐵) = 𝑃(𝐴)
A and/or B occur jointly (outcomes overlap) 𝑃(𝐴 ∪ 𝐵) = 𝑃(𝐴) + 𝑃(𝐵) − 𝑃(𝐴 ∩ 𝐵)
Disjoint events (no outcomes overlap since events do not occur at the same time a.k.a. mutually exclusive events)
A and/or B occur disjoint 𝑃(𝐴 ∪ 𝐵) = 𝑃(𝐴) + 𝑃(𝐵)
A and B occur disjoint 𝑃(𝐴 ∩ 𝐵) = ∅ = 0
A and B do not occur jointly 𝑃(𝐴 ∩ 𝐵)𝐶 = 1 − 𝑃(𝐴 ∩ 𝐵)
Do not occur
A does not occur 𝑃(𝐴𝐶 ) = 1 − 𝑃(𝐴)
None of the three events A, B and C occurs 𝑃(𝐴𝐶 ∩ 𝐵𝐶 ∩ 𝐶 𝐶 )
Whenever TI-30XB MultiView Texas Instruments displays fractions, exact square root or exact pi answers, then press [⊲⊳] to
toggle the display to decimal answers.
Random Variables
A random variable (rv), denoted as 𝑋, is a prescript for measuring some feature of interest of the
sample space; it assigns a value (usually numbers) to each outcome of the experiment, but its actual
outcome is not known in advance (random) and is determined by chance. The actual outcome of a
random variable is called the realisation, denoted as 𝑥, small letters.
Note: A random variable is assumed independent if you are not given more information.
There are two types of random variables: quantitative (ordinary numbers as values) and qualitative
(categories as values). A random variable can be discrete or continuous. Discrete variables have finite
or countable number of outcomes. Continuous variables can take on any value in an interval.
Probability Density Function
Probability Density Function (pdf), 𝑓 shows the probabilities for each possible outcome of a discrete
random variable. Hence: 𝑓(𝑥) = 𝑃(𝑋 = 𝑥) for all outcomes 𝑥 of 𝑋. Two properties:
i. 𝑓(𝑥) ≥ 0 𝑓𝑜𝑟 𝑎𝑙𝑙 𝑥
ii. total surface equals 1
, (Cumulative) Distribution Function
(Cumulative) Distribution Function (cdf), F, Φ always adds up the probabilities per outcome.
Two definitions:
• cdf discrete rv: 𝐹(𝑎) = 𝑃(𝑋 ≤ 𝑎) for all real numbers 𝑎
• cdf continuous rv: 𝐹(𝑎) = 𝑃(𝑋 ≤ 𝑎) = 𝑃(𝑋 < 𝑎) for all real numbers 𝑎
Important properties:
• 𝐹 is non-decreasing
• 𝐹(−∞) = 0; 𝐹(∞) = 1
• 𝐹(𝑏) − 𝐹(𝑎) = 𝑃(𝑋 ≤ 𝑏) − 𝑃(𝑋 ≤ 𝑎) = 𝑃(𝑎 < 𝑋 ≤ 𝑏) for all 𝑎 and 𝑏
Note: Rounded brackets do not include given numbers, in which squared brackets do. Hence: 𝑃([35, ∞)) = 𝑃(𝑋 ≥ 35).
The table of the pdf and cdf:
𝑥 0 1 2
𝑓(𝑥) 0.25 0.50 0.25
𝐹(𝑥) 0.25 0.75 1
Four scenarios:
1. 𝑃(𝑋 > 1) = 𝑃(𝑋 = 2) = 0.25
2. 𝑃(𝑋 < 2) = 𝑃(𝑋 = 0) + 𝑃(𝑋 = 1) = 1 − 𝑃(𝑋 = 2) = 0.75
3. 𝑃(𝑋 ≥ 0) = 𝑃(𝑋 = 0) + 𝑃(𝑋 = 1) + 𝑃(𝑋 = 2) = 1
4. 𝑃(𝑋 ≤ 1) = 𝑃(𝑋 = 0) + 𝑃(𝑋 = 1) = 0.75
The graph of the cdf of 𝑋: left continuous and right discrete.
Expectation, Variance and Standard Deviation
Chance is driven by the probability distribution of 𝑋. The actual outcome will deviate from that
expectation.
Expectation, 𝐸(𝑋): 𝜇𝑥 = ∑𝑥 𝑥 ∗ 𝑓(𝑥) Note: Also called expected value, (weighted) average or mean.
Variance, 𝑉(𝑋): 𝜎 2 = ∑𝑥(𝑥 − 𝜇𝑥 )2 ∗ 𝑓(𝑥) Note: The variance is always non-negative. Hence: 𝑉(𝑋) ≥ 0
2
Shortcut formula: 𝜎 2 = (∑𝑥 𝑥 2 ∗ 𝑓(𝑥)) − 𝜇𝑥2 = 𝐸(𝑋 2 ) − (𝐸(𝑋))
Standard Deviation, 𝑆𝐷(𝑋): 𝜎 = √𝑉(𝑋) Note: Also called mean deviation from averages or mean spread.
