Summary for Linear algebra and applications (2DBI00)

LINEAR ALGEBRA AND APPLICATIONS
OLD LECTURES

https://intranet.tue.nl/onderwijs/videocollege-tue/
2018/2019:
Mathematics → 2DBI00 → 2018-2019
SETUP

Exam 60%
Group assignment 20%
SOWISO 25 questions 20% Similar to calculus, you can repeat to get 100%

COURSE SCHEDULE

1. Vectors and matrices
2. Vector and matrix norms
3. Google Pagerank, Markov chains
4. Linear systems
5. Linear algebra on computers
6. Inverse matrices
7. Determinants
8. Eigenvalues, eigenvectors
9. Clustering, SPD
10. Orthogonality, basis, rank
11. SVD, data mining
12. Least squares, fitting
13. Rotations, projections
14. Splines, TSVD, low rank

For the exam:
All slides, homework assignments and online assignments
The book is there as reference, most information should be clear from the slides

INTRODUCTION

Linear algebra is fast and elegant, this should be used as much as possible.
Everything non-linear is much slower and harder.

, MATRICES AND VECTORS Matrix × Matrix (𝑨 ∙ 𝒙 = 𝑨𝒙)

Matrix A rectangular table with numbers 𝒄𝒊𝒋 = ∑ 𝒂𝒊𝒌 𝒃𝒌𝒋 = 𝒂𝒊𝟏 𝒃𝟏𝒋 + ⋯ + 𝒂𝒊𝒏 𝒃𝒏𝒋
𝒌=𝟏
(data points are the columns)
𝑎 𝑏 𝑒 𝑓 𝑎𝑒 + 𝑏𝑔 𝑎𝑓 + 𝑏ℎ
Dimensions The size of the matrix [ ]∙[ ] = [𝐴 ∙ 𝐵.1 𝐴 ∙ 𝐵.2 ] = [ ]
𝑐 𝑑 𝑔 ℎ 𝑐𝑒 + 𝑑𝑔 𝑐𝑓 + 𝑑ℎ
(a number is a 1×1 matrix)
Vector A column with numbers (a matrix of ∈ ℝ𝒏×𝟏) 𝑨 ∈ ℝ𝒎×𝒏 × 𝑩 ∈ ℝ𝒏×𝒑 = 𝒚 ∈ ℝ𝒎×𝒑
Indexing 𝑨𝒊𝒋 is the element on the 𝒊th row, and the 𝒋th The inner dimensions (𝑛) must agree, the result has the
column, 𝒙𝒋 is the 𝒋th item in the vector. outside dimensions
Linear if 𝑨(𝒙 + 𝒚) = 𝑨𝒙 + 𝑨𝒚 for all 𝒙, 𝒚 ∈ ℝ𝒏 ,
and 𝑨(𝜶𝒚) = 𝜶𝑨𝒚 for all 𝒚 ∈ ℝ𝒏 and 𝜶 ∈ ℝ.
This is summarized to 𝑨(𝒙 + 𝜶𝒚) = 𝑨𝒙 + 𝜶𝑨 MATRIX-MATRIX MULTIPLICATION

For 𝑨𝟐 , 𝑨 must be a square matrix.
NOTATION The order of multiplication may matter in the cases of
computing time and for rounding errors.
Matrices capital letters 𝑨, 𝑩
Vectors lower case letters 𝒙, 𝒚 / 𝒙, 𝒚
Matrix-matrix multiplication is …
Constants lower case Greek letters 𝜶, 𝜷
Matrix size 𝑨 ∈ ℝ𝒏×𝒎 (𝒏 = rows and 𝒎 = columns) Associative (𝐴𝐵)𝐶 = 𝐴(𝐵𝐶) (𝛼𝐴)𝐵 = 𝛼(𝐴𝐵)
(𝛼𝐴)𝑥 = 𝛼(𝐴𝑥)
𝐴(𝐵𝑥) = (𝐴𝐵)𝑥
Inner product (𝒂 ∙ 𝒃) of vectors 𝒂 and 𝒃
Distributive 𝐴(𝐵 + 𝐶) = 𝐴𝐵 + 𝐴𝐶
𝑻
 = 𝒂 𝒃 = (𝒂𝟏 , … , 𝒂𝒏 )(𝒃𝟏 , … , 𝒃𝒏 ) = 𝒂𝟏 𝒃𝟏 + ⋯ + 𝒂𝒏 𝒃𝒏 Generally not commutative: 𝐴𝐵 ≠ 𝐵𝐴
 Length – ||𝒂|| = √𝒂 ∙ 𝒂 = √𝒂𝟐𝟏 + ⋯+ 𝒂𝟐𝒏
𝒂∙𝒃
Multiplication with diagonal matrices
 Angle – 𝐜𝐨𝐬(𝒂, 𝒃) = ||𝒂||||𝒃||
𝑎 0 𝑐 𝑑 𝑎𝑐 𝑎𝑑
⇒ 𝒂 ∙ 𝒃 = ||𝒂||||𝒃||𝐜𝐨𝐬(𝜶), where 𝜶 = ∠(𝒂, 𝒃) Left multiplication [ ]∙[ ]=[ ]
0 𝑏 𝑒 𝑓 𝑏𝑒 𝑏𝑓
Works on rows
Orthogonal vectors 𝑐 𝑑 𝑎 0 𝑎𝑐 𝑏𝑑
Right multiplication [ ]∙[ ]=[ ]
Orthogonal vectors are perpendicular vectors. 𝑒 𝑓 0 𝑏 𝑎𝑒 𝑏𝑓
 If 𝒂 ∙ 𝒃 = 𝟎, then 𝒂 ⊥ 𝒃 Works on columns
𝟎 𝝅
 If 𝒂 ∙ 𝒃 = 𝟎, 𝐜𝐨𝐬(𝒂, 𝒃) = ||𝒂||||𝒃|| ⇒ ∠(𝒂, 𝒃) =
𝟐
The transpose (𝑨𝑻 )𝒊𝒋 = 𝑨𝒋𝒊
(−𝒃𝒕, 𝒂𝒕) 𝒂𝒏𝒅 (𝒃𝒕, −𝒂𝒕) are perpendicular to (𝒂, 𝒃)
The matrix resulting from switching the rows and columns
 (𝑨 + 𝑩)𝑻 = 𝑨𝑻 + 𝑩𝑻
MATRIX MANIPULATION  (𝑨𝑩)𝑻 = 𝑩𝑻 𝑨𝑻 (𝐴𝐵𝐶)𝑇 = 𝐶 𝑇 𝐵 𝑇 𝐴𝑇
Notice that the order of the matrices changes!
Addition and subtraction  (𝑨𝑻 )𝑻 = 𝑨
 𝑨𝑻 𝑨 and 𝑨 + 𝑨𝑻 are always symmetric
(𝑨 + 𝑩)𝒊𝒋 = 𝑨𝒊𝒋 + 𝑩𝒊𝒋 (𝑨 − 𝑩)𝒊𝒋 = 𝑨𝒊𝒋 − 𝑩𝒊𝒋

𝑎 𝑏 𝑒 𝑓 𝑎+𝑒 𝑏+𝑓
[ ]+[ ]=[ ]
𝑐 𝑑 𝑔 ℎ 𝑐+𝑔 𝑑+ℎ

! This is only possible if 𝑨 and 𝑩 are the same size !


Scalar multiplication
(𝜶𝑨)𝒊𝒋 = 𝜶𝑨𝒊𝒋
𝑎 𝑏 𝛼𝑎 𝛼𝑏
𝛼∙[ ]=[ ]
𝑐 𝑑 𝛼𝑐 𝛼𝑑
(Scalars are real numbers.)


Matrix × Vector (𝑨 ∙ 𝑩 = 𝑨𝑩)
𝒚𝟏 = ∑ 𝒂𝒊𝒋 𝒙𝒋 = 𝒂𝒊𝟏 𝒙𝟏 + ⋯ + 𝒂𝒊𝒏 𝒙𝒏
𝒋=𝟏
𝑎 𝑏 𝑒 𝑎 𝑏 𝑎𝑒 + 𝑏𝑓
[ ]∙[ ] = [ ]∙𝑒+[ ]∙𝑓 = [ ]
𝑐 𝑑 𝑓 𝑐 𝑑 𝑐𝑒 + 𝑑𝑓

𝑨 ∈ ℝ𝒎×𝒏 × 𝒙 ∈ ℝ𝒏 = 𝒚 ∈ ℝ𝒎
When performing vector×matrix computations, the vector
must be a row vector to satisfy dimension requirements.