1 Introduction
Population - The de
ned set of object we desire information from. We can have a population mean µ, median µ̃,
variance σ2 , and standard deviation σ.
Sample - The subset of the population we analyze. We can have a sample
n
mean (arithmetic average of
P
x i
x̄ = n
i=1
n 2
data), median x̃ (the middle value of an ordered data set), variance s2 = with standard deviation s. The
P
i=1 (xi −x̄)
√ n−1
standard error of the mean is s/ n.
Variables are either quantitative, continuous or discrete, or qualitative, categorical (ordinal or nominal) or unstruc-
tured.
A multimodal histogram has multiple peaks. A dichotomous population consists of only two categories; the
sample proportion in category one is x/n, and 1 − x/n in category two.
Counting
Order with replacement: nk . Order without replacement: (n−k)!
n!
. Order with non-distinct objects of K sets: n!
n1 !...nK ! .
No order with replacement: n+k−1
. No order without replacement: nk .
k
2 Probability
An event is a subset of outcomes contained in the sample space which is the set of all possible outcomes.
Union A ∪ B
Intersection A ∩ B
Disjoint Events - when A ∩ B = ∅
Probability of A from the sample: P (A) = cardinality
cardinality S .
A
DeMorgan's Laws (A ∪ B)′ = A′ ∩ B ′ and (A ∩ B)′ P = A′ ∪ B ′ .
Axioms P (A) ≥ 0, P (S) = 1, and P (A1 ∪ A2 ∪ ...) = ∞ i=1 P (Ai ) for a collection of disjoint events.
For any event A, P (A) + P (A′ ) = 1 and P (A) ≤ 1.
Union of Three Events - P (A ∪ B ∪ C) = P (A) + P (B) + P (C) − P (A, B) + P (A, C) + P (B, C) + P (A, B, C).
Conditional Probability - P (A|B) = (A,B)P (B)
Independence - A and B are independent events if: P (A, B) = P (A)P (B), P (A|B) = P (A), or P (B|A) = P (B),
mutual independence extends for k events..
Intersection via Conditioning (Multiplication Rule) - P (A, B) = P (B|A)P (A)
Pk
LoTP - For any set of disjoint and exhaustive events A1 , ..., Ak , P (B) = P (B|A1 )P (A1 ) + · · · + P (B|Ak )P (Ak ) =
i=1 P (B|Ai )P (Ai )
Bayes' Theorem - Let A1 , ..., Ak be any set of disjoint and exhaustive events with prior probabilities P (Ai ). The
posterior probability of Aj given B is P (Aj |B) = P (Aj ,B)
P (B) = Pk
P (B|Aj )P (Aj )
i=1 P (B|Ai )P (Ai )
Random Variables (RV's) and Distributions
RV - A function whose domain is S and whose range is R.
Realization - For some outcome ω and rv X , X(ω) = x is the realization.
PMF - Gives theP probability that a discrete rv takes on x, de
ned as P (X = x) = fX (x). A PMF satis
es
fX (x) ≥ 0 ∀ x and fX (x) = 1.
x
PDF - A function f( x) for a continuous rv such that P (a ≤ X ≤ b) = a f (x)dx. A PDF satis
es f (x) ≥ 0 ∀ x and
Rb
f (x)dx = 1. If the CDF is dierentiable, then the PDF is the derivative of the CDF.
R∞
−∞
Expected Values (of a function of a rv) - (X = x) for discrete X and E[g(X)] =
P
E[g(X)] = x g(x)P
g(x)f (x)dx for continuous X .
R∞
−∞
Linearity of EV's - E[aX + bY + c] = aE[X] + bE[Y ] + c
Variance - V ar[X] = E[X − E[X]]p2 = E[X 2 ] − (E[X])2
Standard Deviation - SD[X] = V ar[X] P
Discrete CDF - FX (x) = P (X ≤ x) = y:y≤x fX (y), the observed value of X will be at most x. Note that
P (a ≤ X ≤ b) = F (b) − F (a−), where F (a−) includes P (X = a) for discrete X .
Continuous CDF - F (x) = P (X ≤ x) = −∞ f (y)dy (integrate the PDF). Note that P (X > a) = 1 − F (a) and
Rx
P (a ≤ X ≤ b) = F (b) − F (a).
Properties of CDFs - limx→−∞ P (X ≤ x) = 0, limx→∞ P (X ≤ x) = 1, P (X ≤ x) is non-decreasing, and P (X ≤ x)
is right continuous.
1
Population - The de
ned set of object we desire information from. We can have a population mean µ, median µ̃,
variance σ2 , and standard deviation σ.
Sample - The subset of the population we analyze. We can have a sample
n
mean (arithmetic average of
P
x i
x̄ = n
i=1
n 2
data), median x̃ (the middle value of an ordered data set), variance s2 = with standard deviation s. The
P
i=1 (xi −x̄)
√ n−1
standard error of the mean is s/ n.
Variables are either quantitative, continuous or discrete, or qualitative, categorical (ordinal or nominal) or unstruc-
tured.
A multimodal histogram has multiple peaks. A dichotomous population consists of only two categories; the
sample proportion in category one is x/n, and 1 − x/n in category two.
Counting
Order with replacement: nk . Order without replacement: (n−k)!
n!
. Order with non-distinct objects of K sets: n!
n1 !...nK ! .
No order with replacement: n+k−1
. No order without replacement: nk .
k
2 Probability
An event is a subset of outcomes contained in the sample space which is the set of all possible outcomes.
Union A ∪ B
Intersection A ∩ B
Disjoint Events - when A ∩ B = ∅
Probability of A from the sample: P (A) = cardinality
cardinality S .
A
DeMorgan's Laws (A ∪ B)′ = A′ ∩ B ′ and (A ∩ B)′ P = A′ ∪ B ′ .
Axioms P (A) ≥ 0, P (S) = 1, and P (A1 ∪ A2 ∪ ...) = ∞ i=1 P (Ai ) for a collection of disjoint events.
For any event A, P (A) + P (A′ ) = 1 and P (A) ≤ 1.
Union of Three Events - P (A ∪ B ∪ C) = P (A) + P (B) + P (C) − P (A, B) + P (A, C) + P (B, C) + P (A, B, C).
Conditional Probability - P (A|B) = (A,B)P (B)
Independence - A and B are independent events if: P (A, B) = P (A)P (B), P (A|B) = P (A), or P (B|A) = P (B),
mutual independence extends for k events..
Intersection via Conditioning (Multiplication Rule) - P (A, B) = P (B|A)P (A)
Pk
LoTP - For any set of disjoint and exhaustive events A1 , ..., Ak , P (B) = P (B|A1 )P (A1 ) + · · · + P (B|Ak )P (Ak ) =
i=1 P (B|Ai )P (Ai )
Bayes' Theorem - Let A1 , ..., Ak be any set of disjoint and exhaustive events with prior probabilities P (Ai ). The
posterior probability of Aj given B is P (Aj |B) = P (Aj ,B)
P (B) = Pk
P (B|Aj )P (Aj )
i=1 P (B|Ai )P (Ai )
Random Variables (RV's) and Distributions
RV - A function whose domain is S and whose range is R.
Realization - For some outcome ω and rv X , X(ω) = x is the realization.
PMF - Gives theP probability that a discrete rv takes on x, de
ned as P (X = x) = fX (x). A PMF satis
es
fX (x) ≥ 0 ∀ x and fX (x) = 1.
x
PDF - A function f( x) for a continuous rv such that P (a ≤ X ≤ b) = a f (x)dx. A PDF satis
es f (x) ≥ 0 ∀ x and
Rb
f (x)dx = 1. If the CDF is dierentiable, then the PDF is the derivative of the CDF.
R∞
−∞
Expected Values (of a function of a rv) - (X = x) for discrete X and E[g(X)] =
P
E[g(X)] = x g(x)P
g(x)f (x)dx for continuous X .
R∞
−∞
Linearity of EV's - E[aX + bY + c] = aE[X] + bE[Y ] + c
Variance - V ar[X] = E[X − E[X]]p2 = E[X 2 ] − (E[X])2
Standard Deviation - SD[X] = V ar[X] P
Discrete CDF - FX (x) = P (X ≤ x) = y:y≤x fX (y), the observed value of X will be at most x. Note that
P (a ≤ X ≤ b) = F (b) − F (a−), where F (a−) includes P (X = a) for discrete X .
Continuous CDF - F (x) = P (X ≤ x) = −∞ f (y)dy (integrate the PDF). Note that P (X > a) = 1 − F (a) and
Rx
P (a ≤ X ≤ b) = F (b) − F (a).
Properties of CDFs - limx→−∞ P (X ≤ x) = 0, limx→∞ P (X ≤ x) = 1, P (X ≤ x) is non-decreasing, and P (X ≤ x)
is right continuous.
1
