Topic Sub-Topic Summary of Key Points & Formulae Key Takeaways!
Matrix Algebra Vectors & A 𝑚𝑥𝑛 matrix ⟶ bold uppercase letters. Key properties of matrices:
Review Matrices; Matrix A vector ⟶ bold lowercase letters. o 𝑨+𝑩 =𝑩+𝑨
Operations; 2 matrices are said to be conformable if they have the appropriate dimensions for the same o (𝑨 + 𝑩) + 𝑪 = 𝑨 + (𝑩 +
Transpose operation. If they are not, it’s not possible to complete the operation. 𝑪)
o Addition/Subtraction: same number of rows & columns. o (𝑨𝑩)𝑪 = 𝑨(𝑩𝑪) = 𝑨𝑩𝑪
o Multiplication: no. of columns for 1st matrix should equal no. of rows for 2nd o 𝑨(𝑩 + 𝑪) = 𝑨𝑩 +
matrix. 𝑨𝑪 𝑎𝑛𝑑 (𝑩 + 𝑪)𝑨 =
𝑩𝑨 + 𝑪𝑨
Key properties & facts!
o 𝑨+𝑩 =𝑩+𝑨 o 𝑨𝑩 not necessarily equal to
o (𝑨 + 𝑩) + 𝑪 = 𝑨 + (𝑩 + 𝑪) 𝑩𝑨
o (𝐴𝐵)𝐶 = 𝐴(𝐵𝐶) = 𝐴𝐵𝐶 o 𝑨𝑩 = 𝟎 possible even if
o 𝐴(𝐵 + 𝐶) = 𝐴𝐵 + 𝐴𝐶 𝑎𝑛𝑑 (𝐵 + 𝐶)𝐴 = 𝐵𝐴 + 𝐶𝐴 𝑨 ≠ 0 and 𝑩 ≠ 0
o 𝐴𝐵 not necessarily equal to 𝐵𝐴 o 𝑪𝑫 = 𝑪𝑬 possible even if
o 𝐴𝐵 = 0 possible even if 𝐴 ≠ 0 and 𝐵 ≠ 0 𝑪 = 0 and 𝑫 ≠ 𝑬
o 𝐶𝐷 = 𝐶𝐸 possible even if 𝐶 = 0 and 𝐷 ≠ 𝐸 Properties of Transposes
Transpose of a 𝑚𝑥𝑛 matrix A is the 𝑛𝑥𝑚 matrix B such that 𝑏 = 𝑎 , ∀ 𝑖 = 1, … , 𝑛 𝑎𝑛𝑑 𝑗 = o (𝑨 ) = 𝑨
o (𝑨 + 𝑩) = 𝑨 + 𝑩
1, … , 𝑛
o (𝑨𝑩) = 𝑩 𝑨
Key properties!
Properties of Inverses:
o (𝑨 ) = 𝑨 𝟏
o (𝑨 + 𝑩) = 𝑨 + 𝑩 o 𝑨 𝟏 =𝑨
𝟏
o (𝑨𝑩) = 𝑩 𝑨 o (𝑨𝑩) = 𝑩 𝟏𝑨 𝟏
Some Special Square Matrix: 𝑚𝑥𝑛 matrix where 𝑚 = 𝑛. o 𝑨𝑻
𝟏
= 𝑨 𝟏 𝑻
Matrices Symmetric Matrix: square matrix 𝑨 where 𝑨 = 𝑨 Properties of Traces (where A,B &
Diagonal Matrix: a symmetric matrix 𝑨 where all off-diagonal terms are 0, i.e., 𝑎 = 0 ∀ 𝑖 ≠ 𝑗 C are nxn matrices and 𝛾 is a
Identity Matrix: A diagonal matrix where all elements on the principal diagonal are 1s. 𝑎 = scalar)
1 ∀ 𝑖 = 1, … , 𝑛 𝑎𝑛𝑑 𝑎 = 0 ∀ 𝑖 ≠ 𝑗. o 𝒕𝒓(𝑨 + 𝑩) = 𝒕𝒓(𝑨) +
o With an identity matrix, 𝑨𝑰 = 𝑰𝑨 = 𝑨 𝒕𝒓(𝑩)
Idempotent Matrix: A square matrix 𝐴 where 𝐴𝐴 = 𝐴. o 𝒕𝒓(𝜸𝑨) = 𝜸𝒕𝒓(𝑨)
Rank; Inverse; Linear Dependence: A set of m-vectors 𝑥 , … , 𝑥 is linearly dependent if there exist numbers o 𝒕𝒓(𝑨𝑩𝑪) = 𝒕𝒓(𝑩𝑪𝑨) =
Trace 𝛾 , … , 𝛾 such that ∑ 𝛾 𝑥 = 0. 𝒕𝒓(𝑪𝑨𝑩)
Here, 𝑥 = 𝑥 + ⋯ + 𝑥 . (can express one vector as a weighted sum of all other Key results from matrix
differentiation!
vectors)
𝒂𝒃 𝒃𝒂
When considering linear dependencies of matrices, can either look at the rows/columns of = =𝒂
𝒃 𝒃
vectors that make up the matrix, and see if they are linearly dependent, i.e.
, 1 2 1 2 𝑨𝒃 𝒃𝑨
𝑋= → 𝑥 = [1 2] 𝑜𝑟 & 𝑥 = [3 4] 𝑜𝑟 = 𝑨 𝒂𝒏𝒅 =𝑨
3 4 3 4 𝒃 𝒃
𝒃 𝑨𝒃
= (𝑨 + 𝑨 )𝒃
Rank (of a set of vectors): The maximum number of linearly independent vectors that can be 𝒃
𝒃 𝑨𝒃
chosen from the set. If A is symmetric, = 𝟐𝑨𝒃
𝒃
o Rank of a set of m+1 vectors is atmost m.
o For any matrix, rank of row vectors = rank of column vectors.
o Non-Singularity (“full rank” matrices): An mxm matrix A is non-singular if the rank of A is
m.
o Singularity: An mxm matrix A is singular if the rank of A is less than m.
Invertability: An mxm matrix A is invertible if there exists an mxm matrix B such that: 𝐴𝐵 =
𝐵𝐴 = 𝐼
o B is called the inverse of A, 𝐵 = 𝐴
o Applicable only for square matrices!
o A is invertible only if it’s square, non-singular.
Key properties!
(𝐴 ) = 𝐴
(𝐴𝐵) = 𝐵 𝐴
(𝐴 ) = (𝐴 )
Inverse of a 2x2 matrix:
𝑎 𝑏 1 𝑑 −𝑏
𝐴= →𝐴 =
𝑐 𝑑 𝑎𝑑 − 𝑏𝑐 −𝑐 𝑎
Inverse of a diagonal matrix
1 0 0
⎡ 1 ⎤
1 0 0
⎢0 0⎥
𝐴= 0 4 0 →𝐴 =⎢ 4 ⎥
0 0 2 ⎢ 1⎥
⎣0 0 2⎦
Trace [tr()]: The sum of the elements on the principal diagonal of a square matrix.
𝑡𝑟(𝐴) = 𝑎
Key properties! (A,B & C are nxn matrices and 𝛾 is a scalar)
o 𝑡𝑟(𝐴 + 𝐵) = 𝑡𝑟(𝐴) + 𝑡𝑟(𝐵)
o 𝑡𝑟(𝛾𝐴) = 𝛾𝑡𝑟(𝐴)
o 𝑡𝑟(𝐴𝐵𝐶) = 𝑡𝑟(𝐵𝐶𝐴) = 𝑡𝑟(𝐶𝐴𝐵)
Matrix Mxm matrix w.r.t scalar → mxm matrix.
Differentiation Scalar w.r.t column vector → mxn matrix.
N row vector w.r.t m column vector → mxn matrix.
Key results!